DeLTA seminar by Ola Rønning

Speaker

Ola Rønning, Department of Computer Science at University of Copenhagen

Title

General and Integrated Stein Variational Inference with EinStein

Abstract

Bayesian inference provides a unified framework for quantifying uncertainty in probabilistic models with latent variables. However, exact inference algorithms generally scale poorly with the dimensionality of the model and the size of the data. To overcome the issue of scaling, the ML community has turned to approximate inference. For the Big Data case, the most prominent method is Variational inference (VI), which uses a simpler parametric model to approximate the target distribution of the latent variables.

In recent years, Stein's method has caught the attention of the ML community as a way to formulate new schemes for performing variational inference. Stein's method provides a fundamental technique for approximating and bounding distances between probability distributions. Of particular interest for inference algorithms is the kernel Stein discrepancy, which has led to two new algorithms: Stein Variational Gradient Descent (SVGD) and Stein Point MCMC. SVGD works by iteratively transporting particles sampled from a simple distribution to the target distribution. In contrast, Stein Point MCMC samples from the target distribution by minimizing the kernel Stein discrepancy. Both techniques suffer from the curse of dimensionality. 

We will discuss algorithms that combine SVGD, Stein point MCMC, and VI to alleviate issues due to high-dimensional models and large data sets. All discussed methods are available in our computational framework EinStein, which is implemented in the deep probabilistic programming language NumPyro. We will draw upon our framework to illustrate key concepts with examples. EinStein is currently freely available under the Aleatory Science organization on Github and will be in the main NumPyro distribution later. The framework is an efficient inference tool for practitioners and a flexible and unified codebase for researchers.

Having discussed the methods within EinStein will point to some open questions about Stein's method that we believe are of interest to the Machine Learning community.

_____________________________
Upcoming DeLTA seminars:

13 September 2021 @ 10:00. Shinji Ito.
-------------------
You can subscribe to DeLTA Seminar mailing list by sending an empty email to delta-seminar-join@list.ku.dk.
Information about DeLTA Seminars is available at DeLTA Lab page: https://sites.google.com/diku.edu/delta