Target-aware Bayesian inference via generalized thermodynamic integration

TL;DR

Target-aware Bayesian inference aims to leverage a known function $f$ to accurately estimate a posterior expectation $I=\mathbb{E}_{\bar{\pi}}[f(\mathbf{x})]$. GTI extends thermodynamic integration by representing $I$ as the difference of two ratios $c_+/Z$ and $c_-/Z$ and performing TI along two paths to $f_+\pi$ and $f_-\pi$, with special handling for strictly positive/negative and generic $f$, including restricted-posteriors and correction factors. The framework unifies TI with TABI, provides acceleration and parallelization strategies, and supports vector-valued outputs; it subsumes standard TI as a special case when $f$ coincides with the likelihood. Numerical experiments on Gaussian and banana-shaped densities show that GTI often achieves lower error than competing target-aware methods, especially in higher dimensions or when the target function has substantial mismatch with the prior, while remaining competitive in easier regimes. Overall, GTI offers a robust, scalable approach for efficient target-aware inference in Bayesian settings, with potential extensions to continuous paths and regression-based density approximations.

Abstract

In Bayesian inference, we are usually interested in the numerical approximation of integrals that are posterior expectations or marginal likelihoods (a.k.a., Bayesian evidence). In this paper, we focus on the computation of the posterior expectation of a function $f(\x)$. We consider a \emph{target-aware} scenario where $f(\x)$ is known in advance and can be exploited in order to improve the estimation of the posterior expectation. In this scenario, this task can be reduced to perform several independent marginal likelihood estimation tasks. The idea of using a path of tempered posterior distributions has been widely applied in the literature for the computation of marginal likelihoods. Thermodynamic integration, path sampling and annealing importance sampling are well-known examples of algorithms belonging to this family of methods. In this work, we introduce a generalized thermodynamic integration (GTI) scheme which is able to perform a target-aware Bayesian inference, i.e., GTI can approximate the posterior expectation of a given function. Several scenarios of application of GTI are discussed and different numerical simulations are provided.

Figures (2)

• Figure 1: Relative squared error of the considered algorithms as a function of number of total likelihood evaluations $E$, for different $y$ and $D$. The median, 25% and 75% quantiles (over 100 independent simulations) are depicted.
• Figure 2: Plots of $\pi({\bf x})$, $f({\bf x})\pi({\bf x})$ and $f({\bf x})^\beta \pi({\bf x})$ with $\beta=0.0173$ for the banana example. We see that $f({\bf x})$ and $f({\bf x})\pi({\bf x})$ have little overlap and hence a direct MCMC estimate of $\mathbb{E}_{\bar{\pi}}[f({\bf x})]$ is not efficient. The tempered distributions, $f({\bf x})^\beta \pi({\bf x})$, are in-between those distributions, helping in the estimation of $\mathbb{E}_{\bar{\pi}}[f({\bf x})]$.