Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Zero variance self-normalized importance sampling via estimating equations

Art B. Owen

TL;DR

This work introduces EE-SNIS, a zero-variance approach to self-normalized importance sampling obtained by rewriting the SNIS target as an estimating equation using Fieller's technique and applying a positivisation trick. By sampling from two distributions $q_+$ and $q_-$ and solving $\Psi_{n_+,n_-}(\mu)=0$, the estimator $\hat{\mu}$ achieves consistency and asymptotic normality with an explicit variance that can be made arbitrarily small through sampler design. Unlike prior SNIS methods, EE-SNIS does not require close approximation to the target density $p$, instead leveraging separate sampling distributions to drive variance toward zero. The paper also discusses existence/uniqueness conditions, potential centering strategies, and alternative zero-variance SNIS schemes, framing a flexible toolkit for variance reduction in challenging Bayesian and rare-event settings.

Abstract

In ordinary importance sampling with a nonnegative integrand there exists an importance sampling strategy with zero variance. Practical sampling strategies are often based on approximating that optimal solution, potentially approaching zero variance. There is a positivisation extension of that method to handle integrands that take both positive and negative values. Self-normalized importance sampling uses a ratio estimate, for which the optimal sampler does not have zero variance and so zero variance cannot even be approached in practice. Strategies that separately estimate the numerator and denominator of that ratio can approach zero variance. This paper develops another zero variance solution for self-normalized importance sampling. The first step is to write the desired expectation as the zero of an estimating equation using Fieller's technique. Then we apply the positivisation strategy to the estimating equation. This paper give conditions for existence and uniqueness of the sample solution to the estimating equation. Then it give conditions for consistency and asymptotic normality and an expression for the asymptotic variance. The sample size multiplied by the variance of the asymptotic formula becomes arbitrarily close to zero for certain sampling strategies.

Zero variance self-normalized importance sampling via estimating equations

TL;DR

This work introduces EE-SNIS, a zero-variance approach to self-normalized importance sampling obtained by rewriting the SNIS target as an estimating equation using Fieller's technique and applying a positivisation trick. By sampling from two distributions and and solving , the estimator achieves consistency and asymptotic normality with an explicit variance that can be made arbitrarily small through sampler design. Unlike prior SNIS methods, EE-SNIS does not require close approximation to the target density , instead leveraging separate sampling distributions to drive variance toward zero. The paper also discusses existence/uniqueness conditions, potential centering strategies, and alternative zero-variance SNIS schemes, framing a flexible toolkit for variance reduction in challenging Bayesian and rare-event settings.

Abstract

In ordinary importance sampling with a nonnegative integrand there exists an importance sampling strategy with zero variance. Practical sampling strategies are often based on approximating that optimal solution, potentially approaching zero variance. There is a positivisation extension of that method to handle integrands that take both positive and negative values. Self-normalized importance sampling uses a ratio estimate, for which the optimal sampler does not have zero variance and so zero variance cannot even be approached in practice. Strategies that separately estimate the numerator and denominator of that ratio can approach zero variance. This paper develops another zero variance solution for self-normalized importance sampling. The first step is to write the desired expectation as the zero of an estimating equation using Fieller's technique. Then we apply the positivisation strategy to the estimating equation. This paper give conditions for existence and uniqueness of the sample solution to the estimating equation. Then it give conditions for consistency and asymptotic normality and an expression for the asymptotic variance. The sample size multiplied by the variance of the asymptotic formula becomes arbitrarily close to zero for certain sampling strategies.

Paper Structure

This paper contains 13 sections, 5 theorems, 45 equations, 1 table.

Table of Contents

  1. Introduction
  2. Notation
  3. Ordinary importance sampling
  4. Self-normalized importance sampling
  5. The positivisation trick
  6. The Fieller trick
  7. Properties of $\hat{\mu}$
  8. Existence and uniqueness
  9. Consistency
  10. Asymptotic variance and central limit theorem
  11. Alternative centering
  12. Other zero variance SNIS estimates
  13. Conclusions

Key Result

Proposition 1

For $n_\pm\geqslant1$ let $\Psi_{{n_+},{n_-}}$ be defined by equation eq:defpsin. Assume that $\max_{\boldsymbol{x}\in S_+}p_u(\boldsymbol{x})>0$ and that $\max_{\boldsymbol{x}\in S_-}p_u(\boldsymbol{x})>0$. Then there exists a value $\hat{\mu}$ with $\Psi_{{n_+},{n_-}}(\hat{\mu})=0$. Let $\overline

Theorems & Definitions (10)

  • Proposition 1
  • proof
  • Proposition 2
  • proof
  • Proposition 3
  • proof
  • Proposition 4
  • proof
  • Theorem 1
  • proof