Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhanced convergence rates of Adaptive Importance Sampling with recycling schemes via quasi-Monte Carlo methods

Jianlong Chen, Jiarui Du, Xiaoqun Wang, Zhijian He

TL;DR

This work enhances adaptive importance sampling by embedding randomized quasi-Monte Carlo into the Modified AMIS framework, enabling recycling of past samples and improved convergence. The authors establish higher-order $L^q$ error bounds through a smoothed projection technique and prove that the RMSE of the RQMC-based MAMIS estimator scales as $\mathcal{O}(\bar{N}_T^{-1+\epsilon})$ for any $\epsilon>0$, while also detailing MC-based rates for comparison. Theoretical contributions include a rigorous convergence analysis of the parameter updates, auxiliary estimators, and their interactions under RQMC; the MC case is also analyzed via Marcinkiewicz–Zygmund inequalities. Empirically, the method shows faster convergence than MC-based counterparts across mixtures of Gaussians, a banana-shaped model, and Bayesian logistic regression, highlighting practical gains for challenging target distributions.

Abstract

This article investigates the integration of quasi-Monte Carlo (QMC) methods using the Adaptive Multiple Importance Sampling (AMIS). Traditional Importance Sampling (IS) often suffers from poor performance since it heavily relies on the choice of the proposal distributions. The AMIS and the Modified version of AMIS (MAMIS) address this by iteratively refining proposal distributions and reusing all past samples through a recycling strategy. We introduce the RQMC methods into the MAMIS, achieving higher convergence rates compared to the Monte Carlo (MC) methods. Our main contributions include a detailed convergence analysis of the MAMIS estimator under randomized QMC (RQMC) sampling. Specifically, we establish the $L^q$ $(q \geq 2)$ error bound for the RQMC-based estimator using a smoothed projection method, which enables us to apply the Hölder's inequality in the error analysis of the RQMC-based MAMIS estimator. As a result, we prove that the root mean square error of the RQMC-based MAMIS estimator converges at a rate of $\mathcal{O}(\bar{N}_T^{-1+ε})$, where $\bar{N}_T$ is the average number of samples used in each step over $T$ iterations, and $ε> 0$ is arbitrarily small. Numerical experiments validate the effectiveness of our method, including mixtures of Gaussians, a banana-shaped model, and Bayesian Logistic regression.

Enhanced convergence rates of Adaptive Importance Sampling with recycling schemes via quasi-Monte Carlo methods

TL;DR

This work enhances adaptive importance sampling by embedding randomized quasi-Monte Carlo into the Modified AMIS framework, enabling recycling of past samples and improved convergence. The authors establish higher-order error bounds through a smoothed projection technique and prove that the RMSE of the RQMC-based MAMIS estimator scales as for any , while also detailing MC-based rates for comparison. Theoretical contributions include a rigorous convergence analysis of the parameter updates, auxiliary estimators, and their interactions under RQMC; the MC case is also analyzed via Marcinkiewicz–Zygmund inequalities. Empirically, the method shows faster convergence than MC-based counterparts across mixtures of Gaussians, a banana-shaped model, and Bayesian logistic regression, highlighting practical gains for challenging target distributions.

Abstract

This article investigates the integration of quasi-Monte Carlo (QMC) methods using the Adaptive Multiple Importance Sampling (AMIS). Traditional Importance Sampling (IS) often suffers from poor performance since it heavily relies on the choice of the proposal distributions. The AMIS and the Modified version of AMIS (MAMIS) address this by iteratively refining proposal distributions and reusing all past samples through a recycling strategy. We introduce the RQMC methods into the MAMIS, achieving higher convergence rates compared to the Monte Carlo (MC) methods. Our main contributions include a detailed convergence analysis of the MAMIS estimator under randomized QMC (RQMC) sampling. Specifically, we establish the error bound for the RQMC-based estimator using a smoothed projection method, which enables us to apply the Hölder's inequality in the error analysis of the RQMC-based MAMIS estimator. As a result, we prove that the root mean square error of the RQMC-based MAMIS estimator converges at a rate of , where is the average number of samples used in each step over iterations, and is arbitrarily small. Numerical experiments validate the effectiveness of our method, including mixtures of Gaussians, a banana-shaped model, and Bayesian Logistic regression.
Paper Structure (20 sections, 13 theorems, 100 equations, 4 figures, 3 algorithms)

This paper contains 20 sections, 13 theorems, 100 equations, 4 figures, 3 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. The modified version of AMIS
  4. Quasi-Monte Carlo methods
  5. Growth condition
  6. $L^q$ Convergence rate of RQMC methods
  7. Convergence rate of the RQMC-based MAMIS estimator
  8. Convergence of the parameters $\widehat{\theta}_t$
  9. On the property of operator $I_{\psi}^{\ast}$
  10. Convergence of the auxiliary estimator
  11. Discrepancy between the MAMIS estimator and the auxiliary estimator
  12. Convergence Rate of the MC-based MAMIS Estimator
  13. Experimental results
  14. Mixture of Gaussians
  15. The case that all $\Sigma_i$ are the same
  16. ...and 5 more sections

Key Result

Lemma 2.1

\newlabellemma:L2-convergence0 Let $\left\{y_{1},\ldots, y_{n}\right\}$ be an RQMC point set used in the estimator $\widehat{I}_{n}(f)$ given by term:RQMCEstimator_ToIntroducePQMC such that each $y_{j}\sim \mathcal{U}[0,1)^d$ and the star discrepancy of $\left\{y_{1},\ldots, y_{n}\right\}$ satisfi where $C$ is a constant independent of $n$. For the function class $G_{e}(M, B, k)$ with order $0<k<

Figures (4)

  • Figure 1: Log-RMSEs for the mixture of three Gaussians example where all $\Sigma_i$ are the same and the dimension of this example is $20$. The RMSEs are computed based on 50 repetitions.
  • Figure 2: Log-RMSEs for the two-dimensional mixture of five Gaussians example where $\Sigma_i$ are different. The RMSEs are computed based on 50 repetitions.
  • Figure 3: Log-RMSEs for the Bayesian Logistic regression example. The RMSEs are computed based on 50 repetitions.
  • Figure 4: Log-RMSEs for the two-dimensional Banana-shaped target example. The RMSEs are computed based on 50 repetitions.

Theorems & Definitions (26)

  • Lemma 2.1
  • Proof 1
  • Theorem 2.2
  • Proof 2
  • Remark 2.3
  • Theorem 3.2
  • Corollary 3.3
  • Remark 3.4
  • Remark 3.5
  • Theorem 3.6
  • ...and 16 more