Publications and Preprints

For a full overview see my Google Scholar page.

• Prospect Pruning: Finding Trainable Weights at Initialization using Meta-Gradients
  Milad Alizadeh, Shyam Tailor, Luisa Zintgraf, Joost van Amersfoort, Sebastian Farquhar, Nic Lane, Yarin Gal
  ICLR 2022 - Code, BibTex

  @article{alizadeh2022prosprect,
    title={Prospect Pruning: Finding Trainable Weights at Initialization using Meta-Gradients},
    author={Alizadeh, Milad and Tailor, Shyam A. and Zintgaf, Luisa M and van Amersfoort, Joost and Farquhar, Sebastian and Lane, Nicholas Donald and Gal, Yarin},
    booktitle={International Conference on Learning Representations (ICLR)},
    year={2022}
  }
  

• Causal-BALD: Deep Bayesian Active Learning of Outcomes to Infer Treatment-Effects from Observational Data
  Andrew Jesson*, Panagiotis Tigas*, Joost van Amersfoort, Andreas Kirsch, Uri Shalit, Yarin Gal
  NeurIPS 2021 - Code, BibTex

  @article{jesson2021causal,
    title={Causal-BALD: Deep Bayesian Active Learning of Outcomes to Infer Treatment-Effects from Observational Data},
    author={Jesson, Andrew and Panagiotis, Tigas and van Amersfoort, Joost and Kirsch, Andreas and Shalit, Uri and Gal, Yarin},
    booktitle={Advances in Neural Information Processing Systems (NeurIPS)},
    year={2021}
  }
  

• On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertainty
  Joost van Amersfoort, Lewis Smith, Andrew Jesson, Oscar Key, Yarin Gal
  ArXiv - Code, BibTex

  @article{van2021on,
    title={On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertainty},
    author={van Amersfoort, Joost and Smith, Lewis and Jesson, Andrew and Key, Oscar and Gal, Yarin},
    journal={arXiv preprint arXiv:2102.11409},
    year={2021}
  }
  

• Can convolutional ResNets approximately preserve input distances? A frequency analysis perspective
  Lewis Smith, Joost van Amersfoort, Haiwen Huang, Sebastian Farquhar, Stephen Roberts, Yarin Gal
  ArXiv - BibTex

  @article{smith2021can,
    title={Can convolutional ResNets approximately preserve input distances? A frequency analysis perspective},
    author={Smith, Lewis and van Amersfoort, Joost and Huang, Haiwen and Roberts, Stephen and Gal, Yarin},
    journal={arXiv preprint arXiv:2106.02469},
    year={2021}
  }
  

• Deterministic Neural Networks with Inductive Biases Capture Epistemic and Aleatoric Uncertainty
  Jishnu Mukhoti*, Andreas Kirsch*, Joost van Amersfoort, Philip Torr, Yarin Gal
  ArXiv - Code, BibTex

  @article{mukhoti2021deterministic,
    title={Deterministic Neural Networks with Appropriate Inductive Biases Capture Epistemic and Aleatoric Uncertainty},
    author={Mukhoti, Jishnu and Kirsch, Andreas and van Amersfoort, Joost and Torr, Philip HS and Gal, Yarin},
    journal={arXiv preprint arXiv:2102.11582},
    year={2021}
  }
  

• Uncertainty Estimation Using a Single Deep Deterministic Neural Network
  Joost van Amersfoort, Lewis Smith, Yee Whye Teh, Yarin Gal
  ICML 2020 - Video, Code, BibTex

  @inproceedings{van2020uncertainty,
    title={Uncertainty Estimation Using a Single Deep Deterministic Neural Network},
    author={van Amersfoort, Joost and Smith, Lewis and Teh, Yee Whye and Gal, Yarin},
    booktitle={International Conference on Machine Learning (ICML)},
    year={2020}
  }

• BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning
  Andreas Kirsch*, Joost van Amersfoort*, Yarin Gal
  NeurIPS 2019 - Blog, Code, Tutorial, BibTex

  @inproceedings{kirsch2019batchbald,
    title={BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning},
    author={Kirsch, Andreas and van Amersfoort, Joost and Gal, Yarin},
    booktitle={Advances in Neural Information Processing Systems (NeurIPS)},
    year={2019}
  }

Workshop Papers

• Bayesian Deep Learning (NeurIPS 2021):
  • On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertainty
  • Decomposing Representations for Deterministic Uncertainty Estimation
• Uncertainty & Robustness in Deep Learning (ICML 2021):
  • Deterministic Neural Networks with Inductive Biases Capture Epistemic and Aleatoric Uncertainty
  • Deep Deterministic Uncertainty for Semantic Segmentation
  • On Pitfalls in OoD Detection: Predictive Entropy Considered Harmful
• The Neglected Assumptions in Causal Inference (ICML 2021):
  • Causal-BALD: Deep Bayesian Active Learning of Outcomes to Infer Treatment-Effects from Observational Data

Blog Posts

• Variational Inference in Sparse Gaussian Processes and Variational Auto-Encoder
  The post explains variational inference in sparse GPs from a VAE perspective.

Open Source - projects and reproductions

For a full overview of all my open source work, see my Github account.

• FastMNIST: The default PyTorch MNIST dataset implementation is slow when used on a modern GPU. This replacement speeds up training by 2-3x.
• Glow in PyTorch: The code reproduces results from "Do Deep Generative Models Know What They Don't Know?" and includes a pretrained model, evaluation notebooks and training code.
• CIFAR-10 baseline script: A self-contained, 150 line script that trains a ResNet-18 to ~94% accuracy on CIFAR-10 using PyTorch. Useful for obtaining a strong baseline with minimal tricks.