Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A novel sequential method for building upper and lower bounds of moments of distributions

Solal Martin, Emilie Chouzenoux, Victor Elvira

TL;DR

Addresses the problem of computing bounds on moments of unnormalized distributions by developing a sequential envelope method based on Gaussian tangent majorants/minorants. The approach constructs piecewise Gaussian envelopes using adaptively chosen tangency points, proving convergence of upper and lower bounds to the true integral under mild density conditions. It specializes to variance estimation in importance sampling and demonstrates competitive numerical performance against state-of-the-art envelope methods, with practical guidelines for implementation. This yields guaranteed, controllable bounds for moment integrals, enabling reliable error control and improved proposal tuning in Bayesian inference.

Abstract

Approximating integrals is a fundamental task in probability theory and statistical inference, and their applied fields of signal processing, and Bayesian learning, as soon as expectations over probability distributions must be computed efficiently and accurately. When these integrals lack closed-form expressions, numerical methods must be used, from the Newton-Cotes formulas and Gaussian quadrature, to Monte Carlo and variational approximation techniques. Despite these numerous tools, few are guaranteed to preserve majoration/minoration inequalities, while this feature is fundamental in certain applications in statistics. In this paper, we focus on the integration problem arising in the estimation of moments of scalar, unnormalized, distributions. We introduce a sequential method for constructing upper and lower bounds on the sought integral. Our approach leverages the majorization-minimization framework to iteratively refine these bounds, in an enveloped principle. The method has proven convergence, and controlled accuracy, under mild conditions. We demonstrate its effectiveness through a detailed numerical example of the estimation of a Monte-Carlo sampler variance in a Bayesian inference problem.

A novel sequential method for building upper and lower bounds of moments of distributions

TL;DR

Addresses the problem of computing bounds on moments of unnormalized distributions by developing a sequential envelope method based on Gaussian tangent majorants/minorants. The approach constructs piecewise Gaussian envelopes using adaptively chosen tangency points, proving convergence of upper and lower bounds to the true integral under mild density conditions. It specializes to variance estimation in importance sampling and demonstrates competitive numerical performance against state-of-the-art envelope methods, with practical guidelines for implementation. This yields guaranteed, controllable bounds for moment integrals, enabling reliable error control and improved proposal tuning in Bayesian inference.

Abstract

Approximating integrals is a fundamental task in probability theory and statistical inference, and their applied fields of signal processing, and Bayesian learning, as soon as expectations over probability distributions must be computed efficiently and accurately. When these integrals lack closed-form expressions, numerical methods must be used, from the Newton-Cotes formulas and Gaussian quadrature, to Monte Carlo and variational approximation techniques. Despite these numerous tools, few are guaranteed to preserve majoration/minoration inequalities, while this feature is fundamental in certain applications in statistics. In this paper, we focus on the integration problem arising in the estimation of moments of scalar, unnormalized, distributions. We introduce a sequential method for constructing upper and lower bounds on the sought integral. Our approach leverages the majorization-minimization framework to iteratively refine these bounds, in an enveloped principle. The method has proven convergence, and controlled accuracy, under mild conditions. We demonstrate its effectiveness through a detailed numerical example of the estimation of a Monte-Carlo sampler variance in a Bayesian inference problem.

Paper Structure

This paper contains 34 sections, 67 equations, 5 figures, 7 tables, 4 algorithms.

Table of Contents

  1. Introduction and Problem Statement
  2. Introduction
  3. Problem Statement
  4. Paper Contributions and Outline
  5. A first construction of integral bounds
  6. Introduction
  7. Gaussian tangent minorants
  8. Gaussian tangent majorants
  9. First bounds and discussion
  10. Sequential refinement of the bounds
  11. Introduction
  12. Minorant/majorant envelopes
  13. Integral approximation
  14. Iterative tangency point selection
  15. Convergence Analysis
  16. ...and 19 more sections

Figures (5)

  • Figure 1: Example of function $\pi$ (blue thick line) and minorant/majorant functions (black/red thin lines) at various tangency points (black/red circles)
  • Figure 2: Example of function $\pi$ (blue thick line), set of tangent minorant/majorant functions (black/red thin lines) at various tangency points, and associated piecewise minorant/majorant function (black/red thick line).
  • Figure 3: Densities $p$ and $p^2/q$ as a function of $x$.
  • Figure 4: Evolution of the bounds along iterations $n$ of Alg. \ref{['algo:tangency']}, until a relative precision of $\tau = 10^{-4}$ is reached, using Algorithm \ref{['algo:tangency']}.
  • Figure 5: Obtained bounds (red, blue) and empirical variance $V_e$ with $N_{\text{runs}} = 10^6$ (left) and MSE on second order moment (right) for different values of $\theta$. Top: $J=10$, $s=1.2$, middle: $J=10, s = 2$, bottom: $J=2$, $s=1.2$.

Theorems & Definitions (5)

  • proof
  • proof
  • proof
  • proof
  • proof