Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Importance Markov Chain

Charly Andral, Randal Douc, Hugo Marival, Christian P. Robert

TL;DR

The paper presents the Importance Markov Chain (IMC), a meta-algorithm that builds an augmented Markov chain from an instrumental MCMC kernel to produce first marginals distributed as the target $\pi$, while retaining key probabilistic guarantees. It establishes LLN, CLT, and geometric ergodicity for the IMC under mild conditions, and develops optimal replication schemes to minimize asymptotic variance. Pseudo-marginal extensions allow unbiased density estimates to be incorporated without sacrificing ergodicity, broadening applicability to settings with intractable densities. Numerical experiments on a Gaussian mixture and normalizing-flow-based independent IMC demonstrate improved effective sample size and practical advantages in memory usage, illustrating the method’s strength in multimodal and high-dimensional targets.

Abstract

The Importance Markov chain is a novel algorithm bridging the gap between rejection sampling and importance sampling, moving from one to the other through a tuning parameter. Based on a modified sample of an instrumental Markov chain targeting an instrumental distribution (typically via a MCMC kernel), the Importance Markov chain produces an extended Markov chain where the marginal distribution of the first component converges to the target distribution. For example, when targeting a multimodal distribution, the instrumental distribution can be chosen as a tempered version of the target which allows the algorithm to explore its modes more efficiently. We obtain a Law of Large Numbers and a Central Limit Theorem as well as geometric ergodicity for this extended kernel under mild assumptions on the instrumental kernel. Computationally, the algorithm is easy to implement and preexisting libraries can be used to sample from the instrumental distribution.

The Importance Markov Chain

TL;DR

The paper presents the Importance Markov Chain (IMC), a meta-algorithm that builds an augmented Markov chain from an instrumental MCMC kernel to produce first marginals distributed as the target , while retaining key probabilistic guarantees. It establishes LLN, CLT, and geometric ergodicity for the IMC under mild conditions, and develops optimal replication schemes to minimize asymptotic variance. Pseudo-marginal extensions allow unbiased density estimates to be incorporated without sacrificing ergodicity, broadening applicability to settings with intractable densities. Numerical experiments on a Gaussian mixture and normalizing-flow-based independent IMC demonstrate improved effective sample size and practical advantages in memory usage, illustrating the method’s strength in multimodal and high-dimensional targets.

Abstract

The Importance Markov chain is a novel algorithm bridging the gap between rejection sampling and importance sampling, moving from one to the other through a tuning parameter. Based on a modified sample of an instrumental Markov chain targeting an instrumental distribution (typically via a MCMC kernel), the Importance Markov chain produces an extended Markov chain where the marginal distribution of the first component converges to the target distribution. For example, when targeting a multimodal distribution, the instrumental distribution can be chosen as a tempered version of the target which allows the algorithm to explore its modes more efficiently. We obtain a Law of Large Numbers and a Central Limit Theorem as well as geometric ergodicity for this extended kernel under mild assumptions on the instrumental kernel. Computationally, the algorithm is easy to implement and preexisting libraries can be used to sample from the instrumental distribution.
Paper Structure (42 sections, 20 theorems, 154 equations, 2 figures, 2 tables, 1 algorithm)

This paper contains 42 sections, 20 theorems, 154 equations, 2 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Notations
  3. The Rejection Markov chain
  4. Formal definition
  5. Application to sampling
  6. The Importance Markov chain
  7. The extended Markov chain
  8. Invariant probability measure
  9. Existence
  10. Uniqueness
  11. Law of large numbers
  12. Central Limit Theorem
  13. Minimizing the asymptotic variance
  14. Optimal choice of the kernel $\tilde{R}$
  15. Optimal upper bound
  16. ...and 27 more sections

Key Result

Proposition 1

Let $Q$ be a Markov kernel on ${\mathsf X} \times \mathcal{X}$ with invariant probability measure $\mu \in \mathsf{M}_1({\mathsf X})$ and let $\rho: {\mathsf X} \to [0,1]$ be a measurable function. Provided that $\mu(\rho)>0$, the probability measure $\nu$ defined by is invariant with respect to $S$, i.e. $\nu S=\nu$.

Figures (2)

  • Figure 1: Analysis of the effect of $\kappa$ on the ESS and the length of the chain.
  • Figure 2: Comparison of the effective sample size between MH, OSR and IMC.

Theorems & Definitions (51)

  • Proposition 1
  • proof
  • Theorem 2
  • Remark 1
  • Remark 2
  • Remark 3
  • Proposition 3
  • proof
  • Proposition 4
  • proof
  • ...and 41 more