A Review of Diffusion-based Simulation-Based Inference: Foundations and Applications in Non-Ideal Data Scenarios

TL;DR

The paper tackles Bayesian parameter inference when likelihoods are intractable by focusing on diffusion-based SBI, which leverages score-based diffusion models to learn conditional posteriors without explicit likelihoods. It presents a rigorous foundation (forward/backward diffusion, score matching, and probabilistic flows) and explains why diffusion approaches can outperform flow-based SBI in robustness and flexibility, especially under non-ideal data conditions. A taxonomy of architectures—conditional, guided, sequential, compositional, and consistency diffusion—is provided, along with a survey of eight cutting-edge works addressing standard, unstructured, missing, and misspecified data, and it discusses open problems and potential geophysical uncertainty applications. The work highlights how diffusion models enable function-space inference, prior adaptation at inference time, and robust handling of real-world data complexities, aiming to advance uncertainty quantification in large-scale geophysical models and similar scientific domains.

Abstract

For complex simulation problems, inferring parameters of scientific interest often precludes the use of classical likelihood-based techniques due to intractable likelihood functions. Simulation-based inference (SBI) methods forego the need for explicit likelihoods by directly utilizing samples from the simulator to learn posterior distributions over parameters $\mathbfθ$ given observed data $\mathbf{x}_{\text{o}}$. Recent work has brought attention to diffusion models -- a type of generative model rooted in score matching and reverse-time stochastic dynamics -- as a flexible framework SBI tasks. This article reviews diffusion-based SBI from first principles to applications in practice. We first recall the mathematical foundations of diffusion modeling (forward noising, reverse-time SDE/ODE, probability flow, and denoising score matching) and explain how conditional scores enable likelihood-free posterior sampling. We then examine where diffusion models address pain points of normalizing flows in neural posterior/likelihood estimation and where they introduce new trade-offs (e.g., iterative sampling costs). The key theme of this review is robustness of diffusion-based SBI in non-ideal conditions common to scientific data: misspecification (mismatch between simulated training data and reality), unstructured or infinite-dimensional observations, and missingness. We synthesize methods spanning foundations drawing from Schrodinger-bridge formulations, conditional and sequential posterior samplers, amortized architectures for unstructured data, and inference-time prior adaptation. Throughout, we adopt consistent notation and emphasize conditions and caveats required for accurate posteriors. The review closes with a discussion of open problems with an eye toward applications of uncertainty quantification for probabilistic geophysical models that may benefit from diffusion-based SBI.

Figures (2)

• Figure 1: Schematic that represents the flow of ideas connecting aforementioned diffusion models, score matching, and simulation-based inference (SBI). The green box and its nodes highlight score matching as the core technique that enables diffusion-based SBI methods. Arrows indicate how different concepts and methods interrelate, illustrating the pathways from foundational score matching principles to practical SBI algorithms.
• Figure 2: An example of irregular data formats. The left figure shows maximum water levels simulated on a high-resolution mesh with approximately 31,000 nodes, while the right figure shows maximum water levels simulated on a lower-resolution mesh with approximately 8,000 nodes. Spatially, the lefthand mesh encompasses the Gulf of Mexico and the southeastern United States, while the righthand mesh focuses on a more localized region to the west of Florida. Both simulations are generated using the ADCIRC model with OWI wind forcing for Hurricane Ian (2022). The differing mesh resolutions lead to observations of varying dimensions and structures, posing significant challenges for traditional SBI methods that require fixed-size inputs.