Basic MCMC - The Metropolis-Hastings Algorithm
In machine learning, we often face computations over random variables that are analytically intractable and are hence forced to use Monte Carlo (MC) approxim...
Recent Posts
BadTorch Part 3: Language Model Decoding and Dropout
In my spare moments for the past few weeks I have been finishing off my “BadTorch” project of re-creating Pytorch’s module-level API for recurrent language m...
Bandit Algorithms (& The Exploration-Exploitation Tradeoff)
RL problems are unique in that RL agents face much greater uncertainty (e.g. about rewards, the environment) than is faced by models in supervised learning. ...
Variational Dropout in Recurrent Models
A workhorse of deep learning is dropout which is typically thought to help limit the extent to which models overfit to training data. However, the question o...
Exploring Tradeoffs Between Safety Metrics with MNIST
Despite huge advances in their capabilities when measured along standard performance dimensions (e.g. recognising images, producing language, forecasting wea...