Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Simulation-Based Inference via Regression Projection and Batched Discrepancies

Arya Farahi, Jonah Rose, Paul Torrey

TL;DR

This work studies a lightweight, likelihood-free inference method that uses a regression-based projection of observed data to build a self-normalized pseudo-posterior via batched simulator discrepancies. It formalizes the construction as an importance-sampling target $oldsymbol{\pi}_{M,\tau}$ and proves Monte Carlo consistency, stability to the surrogate regression, and concentration around an identified set $oldsymbol{\Theta}^$ as the batch size grows and the bandwidth shrinks. The analysis clarifies when the method yields point versus set identification and characterizes the induced identifiability limitations under low-information summaries. Empirical demonstrations on a nonlinear synthetic model and cosmology calibration with the DREAMS suite illustrate substantial computational gains and the practical trade-offs between identifiability and interpretability.

Abstract

We analyze a lightweight simulation-based inference method that infers simulator parameters using only a regression-based projection of the observed data. After fitting a surrogate linear regression once, the procedure simulates small batches at the proposed parameter values and assigns kernel weights based on the resulting batch-residual discrepancy, producing a self-normalized pseudo-posterior that is simple, parallelizable, and requires access only to the fitted regression coefficients rather than raw observations. We formalize the construction as an importance-sampling approximation to a population target that averages over simulator randomness, prove consistency as the number of parameter draws grows, and establish stability in estimating the surrogate regression from finite samples. We then characterize the asymptotic concentration as the batch size increases and the bandwidth shrinks, showing that the pseudo-posterior concentrates on an identified set determined by the chosen projection, thereby clarifying when the method yields point versus set identification. Experiments on a tractable nonlinear model and on a cosmological calibration task using the DREAMS simulation suite illustrate the computational advantages of regression-based projections and the identifiability limitations arising from low-information summaries.

Simulation-Based Inference via Regression Projection and Batched Discrepancies

TL;DR

This work studies a lightweight, likelihood-free inference method that uses a regression-based projection of observed data to build a self-normalized pseudo-posterior via batched simulator discrepancies. It formalizes the construction as an importance-sampling target and proves Monte Carlo consistency, stability to the surrogate regression, and concentration around an identified set as the batch size grows and the bandwidth shrinks. The analysis clarifies when the method yields point versus set identification and characterizes the induced identifiability limitations under low-information summaries. Empirical demonstrations on a nonlinear synthetic model and cosmology calibration with the DREAMS suite illustrate substantial computational gains and the practical trade-offs between identifiability and interpretability.

Abstract

We analyze a lightweight simulation-based inference method that infers simulator parameters using only a regression-based projection of the observed data. After fitting a surrogate linear regression once, the procedure simulates small batches at the proposed parameter values and assigns kernel weights based on the resulting batch-residual discrepancy, producing a self-normalized pseudo-posterior that is simple, parallelizable, and requires access only to the fitted regression coefficients rather than raw observations. We formalize the construction as an importance-sampling approximation to a population target that averages over simulator randomness, prove consistency as the number of parameter draws grows, and establish stability in estimating the surrogate regression from finite samples. We then characterize the asymptotic concentration as the batch size increases and the bandwidth shrinks, showing that the pseudo-posterior concentrates on an identified set determined by the chosen projection, thereby clarifying when the method yields point versus set identification. Experiments on a tractable nonlinear model and on a cosmological calibration task using the DREAMS simulation suite illustrate the computational advantages of regression-based projections and the identifiability limitations arising from low-information summaries.
Paper Structure (27 sections, 14 theorems, 110 equations, 4 figures)

This paper contains 27 sections, 14 theorems, 110 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Problem Setup
  3. Observed data and augmented predictors.
  4. Surrogate linear model (OLS on observed data).
  5. Simulator and hierarchical data-generation.
  6. Batched simulation.
  7. Batch discrepancy (residual mean).
  8. Kernel weighting and pseudo-posterior.
  9. Theoretical Results
  10. Moment Structure and Central Limit Theorem
  11. Population Weight Function
  12. Large-$M$ limit.
  13. Monte Carlo Consistency
  14. Stability Guarantees
  15. Identifiability and Concentration Properties
  16. ...and 12 more sections

Key Result

Lemma 3.2

Let $s^2(\theta) = v(\theta)/M$. Under Assumption ass:moments, Moreover, for any bounded continuous function $\varphi$,

Figures (4)

  • Figure 1: A realization of simulated observed data.
  • Figure 2: Left Panel: Prior samples in $(\theta_0, \theta_1)$-space (gray), together with the support of the SBI pseudo-posterior induced by batched residual weighting (orange). The prior is diffuse and isotropic, while the pseudo-posterior concentrates along a curved one-dimensional manifold (identified set). Panel Right: Comparison between the exact posterior obtained via Metropolis--Hastings (blue), the SBI pseudo-posterior (orange), and the true parameter value $\theta_{\mathrm{true}}$ (pink cross). The pseudo-posterior exhibits strong concentration near the true posterior mass in one dimension while in the other dimension the identification is not possible.
  • Figure 3: Marginalized 2D pseudo-posterior distributions for the three uncertain astrophysical parameters: the specific energy of supernova feedback ($\bar{e}_w$), the speed of supernova winds ($\kappa_w$), and the AGN feedback coupling strength ($\epsilon_{f,\mathrm{high}}$). From left to right, the columns display constraints derived from the SMHM relation at $z=0$, the SMHM relation at $z=1$, the black hole mass--stellar velocity dispersion ($M$--$\sigma$) relation, and the combined constraints. Brighter colors indicate regions of higher weight, and the cyan annuli indicate the fiducial parameters used in this model. The degeneracy between $\bar{e}_w$ and $\kappa_w$ visible in the $z=0$ SMHM relation is broken by the inclusion of the $z=1$ data.
  • Figure 4: Comparison of simulated galaxy scaling relations with observational data before and after applying pseudo-posterior weights. The panels display the $z=0$ SMHM relation (left), the $z=1$ SMHM relation (middle), and the $M$--$\sigma$ relation (right). The red shaded regions represent the unweighted distribution of the simulations and the blue shaded regions show the distributions after applying the combined weights shown in the rightmost panel of Figure \ref{['fig:constraints']}. The black dashed lines indicate the mean observed relations taken from 2019Behroozi and 2020Greene. The weighted distributions, particularly the $z=1$ SMHM and $M$--$\sigma$ relations, shift significantly toward the observations, demonstrating that the pseudo-posterior method correctly identifies regions of parameter space that better match the observed relations.

Theorems & Definitions (17)

  • Lemma 3.2
  • Lemma 3.3
  • Lemma 3.4: Uniform convergence of weights
  • Definition 3.5
  • Theorem 3.6: Monte Carlo consistency
  • Proposition 3.7
  • Theorem 3.8
  • Remark 3.9
  • Corollary 3.10: Two-stage convergence
  • Remark 3.11
  • ...and 7 more