Generative Modeling: A Review
Maria Nareklishvili, Nick Polson, Vadim Sokolov
TL;DR
The paper surveys simulation-based posterior inference methods that avoid explicit likelihood evaluation by learning a posterior map $\theta = G(Y, Z)$ with $Z \sim p(Z)$. It presents a unifying generative framework and covers architectures such as ABC, VAEs, ICA, normalizing flows, invertible networks, GANs, diffusion models, and deep fiducial inference, with theoretical grounding from the Noise Outsourcing Theorem. A central theme is quantile-based and likelihood-free approaches that enable fast, scalable posterior sampling and uncertainty quantification, illustrated through a Ebola outbreak application and various numerical examples. The work highlights the practical significance of generative Bayesian computation for high-dimensional outcomes and incomplete models, offering a roadmap for future research in learning flexible, data-driven posterior maps. $p(\theta|Y)$ is approximated via deterministic encoders/decoders and latent-variable mappings, enabling efficient posterior draws without repeated likelihood evaluation, which is particularly valuable in complex scientific domains and real-time decision tasks.
Abstract
Generative methods (Gen-AI) are reviewed with a particular goal of solving tasks in machine learning and Bayesian inference. Generative models require one to simulate a large training dataset and to use deep neural networks to solve a supervised learning problem. To do this, we require high-dimensional regression methods and tools for dimensionality reduction (a.k.a. feature selection). The main advantage of Gen-AI methods is their ability to be model-free and to use deep neural networks to estimate conditional densities or posterior quintiles of interest. To illustrate generative methods , we analyze the well-known Ebola data set. Finally, we conclude with directions for future research.