Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Scalable simulation-based inference for implicitly defined models using a metamodel for Monte Carlo log-likelihood estimator

Joonha Park

TL;DR

This paper addresses scalable parameter inference for implicitly defined stochastic models where exact likelihoods are unavailable. It introduces a simulation-based metamodel for the Monte Carlo log-likelihood estimator and proves a local asymptotic normality property for the mean log-likelihood $\mu(\theta;y)$, enabling principled uncertainty quantification. The method yields a MESLE and a simulation-based proxy, with a quadratic metamodel whose curvature matrices $K_1(\theta_0)$ and $K_2(\theta_0)$ drive inference and uncertainty estimates, and it includes automatic tuning and near-optimal design strategies. Across gamma-Poisson, stochastic volatility, and SEIR measles examples, the approach achieves accurate, scalable inference and favorable comparisons to pseudo-marginal MCMC, with practical tools implemented in the R package sbim.

Abstract

Models implicitly defined through a random simulator of a process have become widely used in scientific and industrial applications in recent years. However, simulation-based inference methods for such implicit models, like approximate Bayesian computation (ABC), often scale poorly as data size increases. We develop a scalable inference method for implicitly defined models using a metamodel for the Monte Carlo log-likelihood estimator derived from simulations. This metamodel characterizes both statistical and simulation-based randomness in the distribution of the log-likelihood estimator across different parameter values. Our metamodel-based method quantifies uncertainty in parameter estimation in a principled manner, leveraging the local asymptotic normality of the mean function of the log-likelihood estimator. We apply this method to construct accurate confidence intervals for parameters of partially observed Markov process models where the Monte Carlo log-likelihood estimator is obtained using the bootstrap particle filter. We numerically demonstrate that our method enables accurate and highly scalable parameter inference across several examples, including a mechanistic compartment model for infectious diseases.

Scalable simulation-based inference for implicitly defined models using a metamodel for Monte Carlo log-likelihood estimator

TL;DR

This paper addresses scalable parameter inference for implicitly defined stochastic models where exact likelihoods are unavailable. It introduces a simulation-based metamodel for the Monte Carlo log-likelihood estimator and proves a local asymptotic normality property for the mean log-likelihood , enabling principled uncertainty quantification. The method yields a MESLE and a simulation-based proxy, with a quadratic metamodel whose curvature matrices and drive inference and uncertainty estimates, and it includes automatic tuning and near-optimal design strategies. Across gamma-Poisson, stochastic volatility, and SEIR measles examples, the approach achieves accurate, scalable inference and favorable comparisons to pseudo-marginal MCMC, with practical tools implemented in the R package sbim.

Abstract

Models implicitly defined through a random simulator of a process have become widely used in scientific and industrial applications in recent years. However, simulation-based inference methods for such implicit models, like approximate Bayesian computation (ABC), often scale poorly as data size increases. We develop a scalable inference method for implicitly defined models using a metamodel for the Monte Carlo log-likelihood estimator derived from simulations. This metamodel characterizes both statistical and simulation-based randomness in the distribution of the log-likelihood estimator across different parameter values. Our metamodel-based method quantifies uncertainty in parameter estimation in a principled manner, leveraging the local asymptotic normality of the mean function of the log-likelihood estimator. We apply this method to construct accurate confidence intervals for parameters of partially observed Markov process models where the Monte Carlo log-likelihood estimator is obtained using the bootstrap particle filter. We numerically demonstrate that our method enables accurate and highly scalable parameter inference across several examples, including a mechanistic compartment model for infectious diseases.
Paper Structure (21 sections, 7 theorems, 98 equations, 12 figures, 3 tables, 5 algorithms)

This paper contains 21 sections, 7 theorems, 98 equations, 12 figures, 3 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Simulation-based inference
  3. Simulation-based proxy
  4. Local asymptotic normality for $\mu(\theta;y_{1;n})$
  5. Simulation metamodel
  6. Application to partially observed Markov processes (POMPs)
  7. Bias in simulation-based inference
  8. Estimation of $\theta_\mathit{MESLE}$ and $K_1(\theta_0)$
  9. Estimation of the metamodel parameters
  10. Estimation and uncertainty quantification for the MESLE
  11. Estimation of $K_1(\theta_0)$
  12. Estimation and uncertainty quantification for the simulation-based proxy $\theta_*$
  13. Numerical results
  14. Gamma process with Poisson observations
  15. Stochastic volatility model
  16. ...and 6 more sections

Key Result

Proposition 1

Suppose that Assumptions assum:simll_sum-assum:3rd_diff hold where $Y_{1:n} \sim P^Y_{\theta_0}$. Let Then $S_n$ converges in distribution to $\mathcal{N}(0, K_1(\theta_0))$ and converges in probability to zero uniformly for $t\in B$ as $n\to\infty$ for every bounded set $B$ containing 0.

Figures (12)

  • Figure 1: The top panel shows the likelihood estimates obtained by running the particle filter five times for each $\theta$ value on a natural (non-logarithmic) scale. The bottom panel shows the log of those likelihood estimates, marked by '+' symbols. The maximum among the five values for each $\theta$ is indicated by a larger '+' symbol. The dashed curve shows the fitted quadratic polynomial. The exact log-likelihoods computed using the Kalman filter are indicated by '$\bullet$'. The dotted vertical line indicates the true parameter value, $\theta=0.02$, and the gray vertical lines indicate the constructed 90% and 95% confidence intervals.
  • Figure 2: Simulated log-likelihoods for the gamma-Poisson process. The 90% and 95% confidence intervals constructed for the simulation-based proxy $\lambda_*$ are marked by gray vertical lines. The true value of $\lambda$ is marked by the vertical dashed line. The blue dashed curve indicates the fitted quadratic polynomial, and the red dashed curve the exact log-likelihood function, with a vertical shift for better visual comparison.
  • Figure 3: The left panel shows the distribution of the estimate $\hat{\lambda}_\mathit{MESLE}$ over 10000 replications of simulation-based estimation. The exact value of $\lambda_\mathit{MESLE}$ is marked by the vertical dashed line. A small number of $\hat{\lambda}_\mathit{MESLE}$ that were outside the displayed range were omitted from the plot. The right plot shows the estimated rejection probabilities for $H_0: \lambda_\mathit{MESLE} = \lambda_{H_0}$ at a 5% significance level for varied null values $\lambda_{H_0}$. The blue horizontal line indicates the significance level.
  • Figure 4: The distribution of the estimates for the simulation-based proxy $\lambda_*$ (left) and the estimates for $K_1$ (right). The dashed lines indicate the true values of $\lambda$ and $K_1$.
  • Figure 5: The left panel shows the probability of rejecting the null hypothesis $H_0: \lambda_* = \lambda_{*,0}$ at a 5% significance level for varied null values $\lambda_{*,0}$. The right panel shows the probability of rejecting the null hypothesis when the exact value of $K_1(\lambda_0)$ is used instead of an estimated value.
  • ...and 7 more figures

Theorems & Definitions (13)

  • Example 1
  • Definition 1: MESLE
  • Definition 2: Simulation-based proxy
  • Proposition 1: Local asymptotic normality (LAN) for $\mu(\theta;Y_{1:n})$
  • Definition 3: Conditional simulation metamodel given data
  • Definition 4: Marginal simulation metamodel
  • Proposition 2
  • Proposition 3
  • Proposition 4
  • Corollary 1
  • ...and 3 more