Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Amortised and provably-robust simulation-based inference

Ayush Bharti, Charita Dellaporta, Yuga Hikida, François-Xavier Briol

TL;DR

This work tackles the brittleness of likelihood-free SBI under outliers and misspecification by introducing Neural Score-Matching Bayes (NSM-Bayes), which fuses a neural conditional density surrogate with a weighted score-matching loss inside Generalised Bayesian Inference. It offers two implementations: a general NSM-Bayes and a conjugate NSM-Bayes-conj that yields closed-form posterior updates for Gaussian priors, enabling fully amortised inference. The authors prove robustness guarantees for NSM-Bayes and demonstrate empirical strength across g-and-k, SIR with under-reporting, and radio propagation models, showing superior calibration, robustness, and competitive runtime relative to robust SBI baselines. The approach scales to non-differentiable simulators and leverages IMQ weighting to down-weight outliers, with practical calibration of the learning rate β to match nominal coverage. Limitations include challenges with highly multimodal likelihoods and discrete data, suggesting avenues for extending score-matching robustness to broader SBI settings.

Abstract

Complex simulator-based models are now routinely used to perform inference across the sciences and engineering, but existing inference methods are often unable to account for outliers and other extreme values in data which occur due to faulty measurement instruments or human error. In this paper, we introduce a novel approach to simulation-based inference grounded in generalised Bayesian inference and a neural approximation of a weighted score-matching loss. This leads to a method that is both amortised and provably robust to outliers, a combination not achieved by existing approaches. Furthermore, through a carefully chosen conditional density model, we demonstrate that inference can be further simplified and performed without the need for Markov chain Monte Carlo sampling, thereby offering significant computational advantages, with complexity that is only a small fraction of that of current state-of-the-art approaches.

Amortised and provably-robust simulation-based inference

TL;DR

This work tackles the brittleness of likelihood-free SBI under outliers and misspecification by introducing Neural Score-Matching Bayes (NSM-Bayes), which fuses a neural conditional density surrogate with a weighted score-matching loss inside Generalised Bayesian Inference. It offers two implementations: a general NSM-Bayes and a conjugate NSM-Bayes-conj that yields closed-form posterior updates for Gaussian priors, enabling fully amortised inference. The authors prove robustness guarantees for NSM-Bayes and demonstrate empirical strength across g-and-k, SIR with under-reporting, and radio propagation models, showing superior calibration, robustness, and competitive runtime relative to robust SBI baselines. The approach scales to non-differentiable simulators and leverages IMQ weighting to down-weight outliers, with practical calibration of the learning rate β to match nominal coverage. Limitations include challenges with highly multimodal likelihoods and discrete data, suggesting avenues for extending score-matching robustness to broader SBI settings.

Abstract

Complex simulator-based models are now routinely used to perform inference across the sciences and engineering, but existing inference methods are often unable to account for outliers and other extreme values in data which occur due to faulty measurement instruments or human error. In this paper, we introduce a novel approach to simulation-based inference grounded in generalised Bayesian inference and a neural approximation of a weighted score-matching loss. This leads to a method that is both amortised and provably robust to outliers, a combination not achieved by existing approaches. Furthermore, through a carefully chosen conditional density model, we demonstrate that inference can be further simplified and performed without the need for Markov chain Monte Carlo sampling, thereby offering significant computational advantages, with complexity that is only a small fraction of that of current state-of-the-art approaches.
Paper Structure (54 sections, 8 theorems, 57 equations, 12 figures, 5 tables, 3 algorithms)

This paper contains 54 sections, 8 theorems, 57 equations, 12 figures, 5 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Approximate Bayesian Inference for Simulators
  4. Generalised Bayesian Inference for Simulators
  5. Methodology
  6. Neural Score-matching Bayes for Simulation-based Inference
  7. Conjugate Neural Score-matching Bayes
  8. Hyperparameter Selection
  9. Robustness
  10. Lack of Robustness of Neural Likelihood Estimation
  11. Robustness of Neural Score-matching Bayes
  12. Robustness with Mixture Density Networks and Normalising Flows
  13. Mixture density network.
  14. Masked autoregressive flows.
  15. Robustness of Conjugate Neural Score-matching Bayes
  16. ...and 39 more sections

Key Result

Proposition 1

Suppose assumption:exptheta holds, $\pi(\theta) \propto \mathcal{N}(\theta;\mu,\Sigma)$, and $W(x) = I_{d_\mathcal{X}} w(x)$ for some $w: \mathcal{X} \rightarrow \mathbb{R}$. Then $\pi_{\text{NSM}}(\theta|x_{1:n}^o, \hat{\phi}_m) \propto \mathcal{N}(\theta;\mu_{n,m},\Sigma_{n,m})$ where:

Figures (12)

  • Figure 1: Amortised SBI in the presence of outliers. With a few one-sided outliers (10%) in the observed data ( ), shown in the top-right plot, the neural likelihood estimation (NLE ) posterior of the g-and-k distribution is significantly impacted, specifically parameters $\theta_2$ and $\theta_3$ that govern the scale and skewness of the distribution (details in \ref{['sec:gnk']}). In contrast, our proposed method, named neural score-matching Bayes ( ), is robust as we use the function $w(x)$ ( ) to down-weight the effect of the outliers.
  • Figure 2: The g-and-k distribution. Kernel density estimates of the marginal posterior distributions from NLE ( ), NSM-Bayes ( ), NSM-Bayes-conj ( ), ACE ( ), NPL-MMD ( ), NPE-RS ( ), RSNL ( ), and GBI-SR ( ) from 500 samples, along with the reference NLE posterior ( ) given uncorrupted data. The dashed and dotted gray lines denote the true parameter value and the prior distribution, respectively.
  • Figure 3: Observed uncorrupted ( ) and undercounted ( ) trajectories of incidence from the SIR model with $\epsilon=5\%$ and $50\%$ retention probability.
  • Figure 4: SIR model. Performance vs. posterior inference time for NLE ( ), NSM-Bayes ( ), NSM-Bayes-conj ( ), GBI-SR ( ), and ACE ( ) under both undercounting and heavy-tailed contamination. In both cases, NSM-Bayes-conj performs the best whilst incurring the least time.
  • Figure 4: Ablation study on NSM-Bayes-conj method involving the estimators used in the IMQ weight function $w(x)$.
  • ...and 7 more figures

Theorems & Definitions (16)

  • Proposition 1
  • Theorem 1
  • Theorem 2
  • Corollary 1
  • Proposition 2
  • Proposition 3
  • Theorem 3
  • proof
  • proof
  • proof
  • ...and 6 more