A Primer on Variational Inference for Physics-Informed Deep Generative Modelling

TL;DR

This work presents a comprehensive tutorial and survey on using variational inference (VI) to perform uncertainty quantification for physics-based forward and inverse problems. It builds a unified VI framework that includes Bayes VI and probabilistic generative models and shows how to embed physics through forward maps, PDE residuals, and deep priors. The paper details forward-model-based, residual-based, and dynamical latent approaches, with concrete ELBO formulations, reparameterization tricks, and advanced priors such as deep generative priors and normalizing flows. It also discusses practical considerations, limitations, and future directions, including alternative divergences and small-data regimes, to advance scalable, uncertainty-aware physics-informed inference.

Abstract

Variational inference (VI) is a computationally efficient and scalable methodology for approximate Bayesian inference. It strikes a balance between accuracy of uncertainty quantification and practical tractability. It excels at generative modelling and inversion tasks due to its built-in Bayesian regularisation and flexibility, essential qualities for physics related problems. For such problems, the underlying physical model determines the dependence between variables of interest, which in turn will require a tailored derivation for the central VI learning objective. Furthermore, in many physical inference applications this structure has rich meaning and is essential for accurately capturing the dynamics of interest. In this paper, we provide an accessible and thorough technical introduction to VI for forward and inverse problems, guiding the reader through standard derivations of the VI framework and how it can best be realized through deep learning. We then review and unify recent literature exemplifying the flexibility allowed by VI. This paper is designed for a general scientific audience looking to solve physics-based problems with an emphasis on uncertainty quantification

Figures (1)

• Figure 1: A depiction of the three spaces of inferential interest, the observation space $\mathcal{Y}$, the discretised solution space $\mathcal{U}_h$, and the discretised parameter space $\mathcal{Z}_h$. More specifically, we have an observation $\mathbf{y}\in \mathcal{Y}\subseteq\mathbb{R}^{d_y}$, a solution $u_h\in\mathcal{U}_h\subset\mathcal{U}$, and a parameter $z_h\in\mathcal{Z}_h\subset\mathcal{Z}$.