Optimality in importance sampling: a gentle survey

TL;DR

This survey synthesizes the notion of optimality in importance sampling across static and adaptive settings, including unique and multiple proposal densities, noisy evaluations, and evidence estimation. It derives explicit forms for optimal proposal densities under various objectives (e.g., minimizing IS variance, SNIS variance, or joint estimates of integrals and normalizing constants), and it highlights practical iterative schemes when key quantities are unknown. The work connects standard IS, SNIS, MIS, RIS, umbrella sampling, and bridge sampling, providing theoretical MSE bounds, variance trade-offs, and guidance for constructing low-variance estimators in Bayesian inference tasks such as marginal likelihood estimation and tempered/posterior sequences. Overall, the paper offers a comprehensive framework to tailor proposal densities to specific estimation goals, enabling more efficient Bayesian computation in noisy, multi-target, and model-selection contexts.

Abstract

The performance of the Monte Carlo sampling methods relies on the crucial choice of a proposal density. The notion of optimality is fundamental to design suitable adaptive procedures of the proposal density within Monte Carlo schemes. This work is an exhaustive review around the concept of optimality in importance sampling. Several frameworks are described and analyzed, such as the marginal likelihood approximation for model selection, the use of multiple proposal densities, a sequence of tempered posteriors, and noisy scenarios including the applications to approximate Bayesian computation (ABC) and reinforcement learning, to name a few. Some theoretical and empirical comparisons are also provided.

Figures (6)

• Figure 1: Target density $\bar{\pi}(\theta)$ (blue line) and optimal proposal densities $q_{\text{opt}}(\theta)$ for the standard IS (red line) and SNIS (magenta line) schemes, when $f(\theta)=\theta$.
• Figure 2: Target density $\bar{\pi}(\theta)$ (blue line) and optimal proposal densities $q_{\text{opt}}(\theta)$ for the standard IS (red line) and SNIS (magenta line) schemes, when $f(\theta)=\sqrt{|\theta|}$.
• Figure 3: Target density $\bar{\pi}(\theta)$ (blue line) and optimal proposal densities $q_{\text{opt}}(\theta)$ for the standard IS (red line) and SNIS (magenta line) schemes, when $f(\theta)=\theta^2$.
• Figure 4: The variances $\text{Var}_{\bar{\pi}}[\widehat{Z}_\text{IS}]$ and $\text{Var}_{\bar{\pi}}[1/\widehat{Z}_\text{RIS}]$ in Eqs. \ref{['VAR_IS_teo']} and \ref{['VAR_RIS_inv']}, respectively ($N=500$).
• Figure 5: (a) Bias (dashed line) and variance (solid line) of $\widehat{Z}_\text{RIS}$ as a function of $h$ ($N=500$). (b) MSE of $\widehat{Z}_\text{IS}$ (solid line) and $\widehat{Z}_\text{RIS}$ (dashed line) as a function of $h$ ($N=500$).