Simulation-based Bayesian inference with ameliorative learned summary statistics -- Part I
Getachew K. Befekadu
TL;DR
This paper tackles simulation-based Bayesian inference when the exact likelihood is intractable by introducing learned summary statistics that act as an empirical-likelihood proxy. It builds a transformation framework using the Cressie-Read discrepancy under moment restrictions to connect observed data and simulated outputs, enabling conditioning on observed samples and extending to weakly dependent data. The authors develop multiple extensions, including replication averaging, conditional moment restrictions, and block-wise handling for time-series data, with closed-form discrepancy weights derived via Lagrange multipliers to support scalable, distributed computation and MCMC-based exploration of the parameter space. Part 1 focuses on formal problem statements and theoretical machinery, with numerical results and distributed-optimization demonstrations reserved for Part 2.
Abstract
This paper, which is Part 1 of a two-part paper series, considers a simulation-based inference with learned summary statistics, in which such a learned summary statistic serves as an empirical-likelihood with ameliorative effects in the Bayesian setting, when the exact likelihood function associated with the observation data and the simulation model is difficult to obtain in a closed form or computationally intractable. In particular, a transformation technique which leverages the Cressie-Read discrepancy criterion under moment restrictions is used for summarizing the learned statistics between the observation data and the simulation outputs, while preserving the statistical power of the inference. Here, such a transformation of data-to-learned summary statistics also allows the simulation outputs to be conditioned on the observation data, so that the inference task can be performed over certain sample sets of the observation data that are considered as an empirical relevance or believed to be particular importance. Moreover, the simulation-based inference framework discussed in this paper can be extended further, and thus handling weakly dependent observation data. Finally, we remark that such an inference framework is suitable for implementation in distributed computing, i.e., computational tasks involving both the data-to-learned summary statistics and the Bayesian inferencing problem can be posed as a unified distributed inference problem that will exploit distributed optimization and MCMC algorithms for supporting large datasets associated with complex simulation models.