Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sequential Neural Score Estimation: Likelihood-Free Inference with Conditional Score Based Diffusion Models

Louis Sharrock, Jack Simons, Song Liu, Mark Beaumont

TL;DR

This work develops Sequential Neural Posterior Score Estimation (SNPSE), a likelihood-free Bayesian inference framework for simulator-based models that uses conditional score-based diffusion models to sample from the posterior $p(\theta|x)$. It integrates a sequential training scheme (SNPSE) to guide simulations toward informative regions of parameter space, with a preferred truncated variant TSNPSE that maintains consistency without explicit density corrections. The methodology is validated on standard SBI benchmarks and a real-world neuroscience problem, showing competitive or superior performance to state-of-the-art amortised and sequential approaches, while highlighting practical considerations such as model flexibility and computational cost. Overall, SNPSE provides a robust, diffusion-based alternative to normalising-flow SBI methods, enabling efficient, scalable likelihood-free inference with principled score-based learning. The combination of forward diffusion, score-matching training, and sequential proposal design yields accurate posterior sampling and demonstrates applicability to complex, high-dimensional simulators across scientific domains.

Abstract

We introduce Sequential Neural Posterior Score Estimation (SNPSE), a score-based method for Bayesian inference in simulator-based models. Our method, inspired by the remarkable success of score-based methods in generative modelling, leverages conditional score-based diffusion models to generate samples from the posterior distribution of interest. The model is trained using an objective function which directly estimates the score of the posterior. We embed the model into a sequential training procedure, which guides simulations using the current approximation of the posterior at the observation of interest, thereby reducing the simulation cost. We also introduce several alternative sequential approaches, and discuss their relative merits. We then validate our method, as well as its amortised, non-sequential, variant on several numerical examples, demonstrating comparable or superior performance to existing state-of-the-art methods such as Sequential Neural Posterior Estimation (SNPE).

Sequential Neural Score Estimation: Likelihood-Free Inference with Conditional Score Based Diffusion Models

TL;DR

This work develops Sequential Neural Posterior Score Estimation (SNPSE), a likelihood-free Bayesian inference framework for simulator-based models that uses conditional score-based diffusion models to sample from the posterior . It integrates a sequential training scheme (SNPSE) to guide simulations toward informative regions of parameter space, with a preferred truncated variant TSNPSE that maintains consistency without explicit density corrections. The methodology is validated on standard SBI benchmarks and a real-world neuroscience problem, showing competitive or superior performance to state-of-the-art amortised and sequential approaches, while highlighting practical considerations such as model flexibility and computational cost. Overall, SNPSE provides a robust, diffusion-based alternative to normalising-flow SBI methods, enabling efficient, scalable likelihood-free inference with principled score-based learning. The combination of forward diffusion, score-matching training, and sequential proposal design yields accurate posterior sampling and demonstrates applicability to complex, high-dimensional simulators across scientific domains.

Abstract

We introduce Sequential Neural Posterior Score Estimation (SNPSE), a score-based method for Bayesian inference in simulator-based models. Our method, inspired by the remarkable success of score-based methods in generative modelling, leverages conditional score-based diffusion models to generate samples from the posterior distribution of interest. The model is trained using an objective function which directly estimates the score of the posterior. We embed the model into a sequential training procedure, which guides simulations using the current approximation of the posterior at the observation of interest, thereby reducing the simulation cost. We also introduce several alternative sequential approaches, and discuss their relative merits. We then validate our method, as well as its amortised, non-sequential, variant on several numerical examples, demonstrating comparable or superior performance to existing state-of-the-art methods such as Sequential Neural Posterior Estimation (SNPE).
Paper Structure (86 sections, 5 theorems, 86 equations, 9 figures, 4 algorithms)

This paper contains 86 sections, 5 theorems, 86 equations, 9 figures, 4 algorithms.

Table of Contents

  1. Introduction
  2. Simulation-Based Inference with Diffusion Models
  3. Simulation-Based Inference
  4. Diffusion Models for Simulation-Based Inference
  5. Sequential Neural Score Estimation
  6. Truncated Approach
  7. Alternative Approaches
  8. SNPSE-A.
  9. SNPSE-B.
  10. SNPSE-C.
  11. Related Work
  12. Simulation-Based Inference
  13. Approximating the Posterior.
  14. Approximating the Likelihood.
  15. Approximating the Likelihood Ratio.
  16. ...and 71 more sections

Key Result

Proposition 3.1

Let $\tilde{p}^{r}(\theta) = \frac{1}{r}\sum_{s=0}^{r-1} \bar{p}^{s}(\theta)$, where $\bar{p}^{0}(\theta) = p(\theta)$ and $\bar{p}^{s}(\theta)$ is defined by eq:truncation for all $s\geq 1$. Suppose that $\Theta_{\mathrm{obs}} \subseteq \mathrm{HPR}_\epsilon({p}_{\psi}^{s}(\theta|x_\mathrm{obs}))$ satisfies $s_{\psi^{\star}}(\theta_t, x_\mathrm{obs}, t) = \nabla_{\theta} \log p_t(\theta_t|x_{\ma

Figures (9)

  • Figure 1: Visualisation of posterior inference using Neural Posterior Score Estimation (NPSE) in the 'Two Moons' experiment. The forward process transforms samples from the target posterior distribution ${p(\theta|x)}$ to a tractable reference distribution. The backward process transports samples from the reference to the target posterior. The backward process depends on the scores ${\nabla_{\theta} \log p_t(\theta|x)}$, which can be estimated using score matching techniques given access to samples ${(\theta,x)\sim p(\theta)p(x|\theta)}$ (see Section \ref{['sec:SGM']}).
  • Figure 2: Results on eight benchmark tasks (non-sequential methods).
  • Figure 3: Results on eight benchmark tasks (sequential methods).
  • Figure 4: Results for the Pyloric experiment.
  • Figure 5: Comparison between NPSE and NLSE on four benchmark tasks.
  • ...and 4 more figures

Theorems & Definitions (10)

  • Proposition 3.1
  • proof
  • Lemma 1.1
  • proof
  • Lemma 1.2
  • proof
  • Theorem 1.3
  • proof
  • Proposition 3.1
  • proof