Diffusion Models in Simulation-Based Inference: A Tutorial Review

TL;DR

The tutorial reviews diffusion models for simulation-based inference (SBI), framing SBI as a principled way to infer latent parameters from simulated and real data when likelihoods are intractable. It explains diffusion-based learning (score matching, flow matching) and how conditioning enables neural posterior, neural likelihood, and joint estimation, with inverse kinematics as a concrete example. The paper catalogues design choices (noise schedules, parameterizations, samplers) and special score-guided techniques (adaptive inference, compositional pooling, structured targets) and demonstrates empirical patterns across four case studies, revealing that no single configuration dominates; performance depends on dimensionality and budget. It highlights the practical benefits of diffusion models for SBI—flexibility, scalability, and fast inference via guidance and compositional scoring—while identifying core challenges in calibration, misspecification, and benchmarking that future work must address. Overall, diffusion models emerge as a versatile, modular framework for SBI that can adapt to complex scientific problems, with guidance and compositionality enabling robust, real-time, and hierarchical inference in diverse domains.

Abstract

Diffusion models have recently emerged as powerful learners for simulation-based inference (SBI), enabling fast and accurate estimation of latent parameters from simulated and real data. Their score-based formulation offers a flexible way to learn conditional or joint distributions over parameters and observations, thereby providing a versatile solution to various modeling problems. In this tutorial review, we synthesize recent developments on diffusion models for SBI, covering design choices for training, inference, and evaluation. We highlight opportunities created by various concepts such as guidance, score composition, flow matching, consistency models, and joint modeling. Furthermore, we discuss how efficiency and statistical accuracy are affected by noise schedules, parameterizations, and samplers. Finally, we illustrate these concepts with case studies across parameter dimensionalities, simulation budgets, and model types, and outline open questions for future research.

Figures (15)

• Figure 1: The three overarching fields whose intersection gives rise to simulation-based inference (SBI). Uncertainty quantification, inverse problems, and Bayesian inference. The ingredients of SBI are (1) a simulator that can generate synthetic observations $\mathbf{y}$ given latent parameters $\boldsymbol{\theta}$; (2) a prior over the latent parameters; and (3) an approximator (e.g., a diffusion model) that plays a role in estimating the posterior distribution of parameters from the observations.
• Figure 2: Organization and reading paths through the paper. Tutorial sections (green) establish foundational concepts and benchmarks, advanced sections (blue) develop technical methodology and notation, and the review section (orange) addresses SBI-specific adaptations of diffusion models. Solid arrows mark the main progression through the paper; dashed arrows indicate optional shortcuts and alternative entry points.
• Figure 3: Conceptual overview of diffusion models for simulation-based inference. Diffusion models can solve canonical tasks in SBI, such as neural posterior estimation (NPE), neural likelihood estimation (NLE), or even joint estimation. They do so by recasting sampling from a complex target distribution into a denoising process that starts with a sample $\mathbf{z}_1$ from a simple noise distribution and progressively removes the noise to arrive at a target sample $\mathbf{z}_0$.
• Figure 4: Inverse kinematics toy example. Each panel shows 1,000 approximate posterior samples representing possible arm configurations $\boldsymbol{\theta}$ given an end-effector position $\mathbf{y}_{\text{obs}}$ (indicated by the red crosshair). The first panel shows the reference distribution estimated with ABC-SMC in pyABCschaelte2022pyabc, while the remaining panels display samples from nine different diffusion models (see \ref{['sec:design']} for more details). The white segments indicate maximum a posteriori estimates. The quality of the posterior samples differs widely between the models, as also captured by differences in the maximum mean discrepancy (MMD) metric computed against the ABC posterior.
• Figure 5: Schematic guided score fields. Panels show the resulting densities for (surrogate) priors $\tilde{p}(\theta)$ (top) and target posteriors $\tilde{p}(\theta \mid \cdot)$ (bottom). Arrows show the score used for inference. Columns differ only by the extra guidance term added to $\nabla\log p(\theta)$ (left to right): classifier-free (baseline), constraints $+\,s_c$ (annulus), prior-adaptive $+\,s_{q/p}$, and compositional $+\,s_{c}+s_{q/p}$.