Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fundamental limits for weighted empirical approximations of tilted distributions

Sarvesh Ravichandran Iyer, Himadri Mandal, Dhruman Gupta, Rushil Gupta, Agniv Bandhyopadhyay, Achal Bassamboo, Varun Gupta, Sandeep Juneja

TL;DR

This work develops a comprehensive KS-distance analysis of self-normalized importance sampling for tilting unknown distributions from finite samples. It shows that, in one dimension, the empirical tilting estimator $R_{n,\theta}$ has fluctuations governed by a Gaussian process $\mathcal{G}_{\theta}$ with variance tied to $M_{\theta}$, and it identifies a polynomial vs. super-polynomial sample-size regime depending on whether the underlying $X$ is bounded. Extending to higher dimensions and the Weibull regime, the authors derive scaling limits where tilted samples concentrate near maximal points, leading to Gaussian, Poisson random measure (PRM), or Weibull/max-type limits, with polynomial sample complexity for bounded cases. In unbounded settings, tilting becomes fundamentally harder: for exponential tails, tilting can fail due to a finite radius of moment generating functions, while for normal-like tails $M_{c_n\theta}$ grows exponentially, yielding a three-way regime of accuracy, PRM-driven behavior, or max-dominated limits. The paper also provides extensive simulations validating the theoretical regimes and discusses implications for rare-event sampling and generative modeling.

Abstract

Consider the task of generating samples from a tilted distribution of a random vector whose underlying distribution is unknown, but samples from it are available. This finds applications in fields such as finance and climate science, and in rare event simulation. In this article, we discuss the asymptotic efficiency of a self-normalized importance sampler of the tilted distribution. We provide a sharp characterization of its accuracy, given the number of samples and the degree of tilt. Our findings reveal a surprising dichotomy: while the number of samples needed to accurately tilt a bounded random vector increases polynomially in the tilt amount, it increases at a super polynomial rate for unbounded distributions.

Fundamental limits for weighted empirical approximations of tilted distributions

TL;DR

This work develops a comprehensive KS-distance analysis of self-normalized importance sampling for tilting unknown distributions from finite samples. It shows that, in one dimension, the empirical tilting estimator has fluctuations governed by a Gaussian process with variance tied to , and it identifies a polynomial vs. super-polynomial sample-size regime depending on whether the underlying is bounded. Extending to higher dimensions and the Weibull regime, the authors derive scaling limits where tilted samples concentrate near maximal points, leading to Gaussian, Poisson random measure (PRM), or Weibull/max-type limits, with polynomial sample complexity for bounded cases. In unbounded settings, tilting becomes fundamentally harder: for exponential tails, tilting can fail due to a finite radius of moment generating functions, while for normal-like tails grows exponentially, yielding a three-way regime of accuracy, PRM-driven behavior, or max-dominated limits. The paper also provides extensive simulations validating the theoretical regimes and discusses implications for rare-event sampling and generative modeling.

Abstract

Consider the task of generating samples from a tilted distribution of a random vector whose underlying distribution is unknown, but samples from it are available. This finds applications in fields such as finance and climate science, and in rare event simulation. In this article, we discuss the asymptotic efficiency of a self-normalized importance sampler of the tilted distribution. We provide a sharp characterization of its accuracy, given the number of samples and the degree of tilt. Our findings reveal a surprising dichotomy: while the number of samples needed to accurately tilt a bounded random vector increases polynomially in the tilt amount, it increases at a super polynomial rate for unbounded distributions.
Paper Structure (35 sections, 43 theorems, 346 equations, 6 figures)

This paper contains 35 sections, 43 theorems, 346 equations, 6 figures.

Table of Contents

  1. INTRODUCTION
  2. TILTING FOR FIXED $\theta$
  3. ACCURATE TILTING IN ONE DIMENSION
  4. TILTING IN THE 1D WEIBULL REGIME
  5. TILTING IN HIGH DIMENSIONS
  6. TILTING IN THE UNBOUNDED SETTING
  7. Tilting Exponential Random Variables
  8. Tilting Normal-like Random Variables
  9. Experiments
  10. Unbounded Random Variables
  11. Bounded Random Variables
  12. CONCLUSION
  13. Proofs for Section 2
  14. Proof of Gaussian limit $\mathcal{G}_{\theta}$ in Theorem \ref{['thm:bd']}
  15. Proof of bounds in Theorem \ref{['thm:bd']}
  16. ...and 20 more sections

Key Result

Theorem 1

There exists a Gaussian random field $\mathcal{G}_{\theta}$ on $\mathbb R$ such that where $Z = \sup_{x}|\mathcal{G}_{\theta}(x)|$. Furthermore, there exist constants $C_1,C_2>0$ independent of $X$ and $\theta$ such that where $M_{\theta}$ is as in (m2tmt2). Furthermore, for all $u>0$.

Figures (6)

  • Figure 1: Exponential tilting of Exp(5) distribution with a sequence of $\left(\theta_i, n_i\right)$ s.t. $M_\theta / n \rightarrow 0$.
  • Figure 2: Exponential tilting of Beta(2, 5) distribution with a sequence of $\left(\theta_i, n_i\right)$ s.t. $M_\theta / n \not\rightarrow 0$.
  • Figure 3: Exponential tilting of Beta(2, 5) distribution with a sequence of $\left(\theta_i, n_i\right)$ s.t. $M_\theta / n \rightarrow 0$.
  • Figure 4: Exponential tilting of Uniform(0, 1) distribution with a sequence of $\left(\theta_i, n_i\right)$ s.t. $M_\theta / n \not\rightarrow 0$.
  • Figure 5: Exponential tilting of Uniform(0, 1) distribution with a sequence of $\left(\theta_i, n_i\right)$ s.t. $M_\theta / n \rightarrow 0$.
  • ...and 1 more figures

Theorems & Definitions (89)

  • Theorem 1
  • Theorem 2
  • Proposition 1
  • Theorem 3
  • Lemma 1
  • Lemma 2
  • Lemma 3: Fischer-Tippett-Gnedenko
  • Lemma 4
  • Theorem 4
  • Lemma 5
  • ...and 79 more