Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Spectral gap bounds for reversible hybrid Gibbs chains

Qian Qin, Nianqiao Ju, Guanyang Wang

TL;DR

This work provides a rigorous framework to bound the convergence rate of reversible hybrid Gibbs chains by relating the spectral gap of the hybrid kernel to that of the exact Gibbs chain and the quality of the conditional samplers used for approximation. Using Dirichlet forms and Markov decomposition, the authors derive general inequalities showing that the hybrid chain’s spectral gap lies within a factor governed by the approximation kernels Q_{i,y}, with tighter bounds when these kernels are PSD. The theory is illustrated through three settings: random-walk Metropolis within Gibbs, blocked coordinate updates, and a hybrid slice sampler, yielding explicit bounds on the ratio of asymptotic variances and guidance on when hybrid schemes are nearly as efficient as exact Gibbs. These results offer practical insights for implementing hybrid Gibbs methods in high-dimensional problems and provide a pathway to quantify trade-offs between computational cost and convergence speed. Overall, the paper advances understanding of hybrid MCMC convergence by linking spectral properties to the quality of conditional approximations and by delivering concrete, applicable bounds.

Abstract

Hybrid Gibbs samplers represent a prominent class of approximated Gibbs algorithms that utilize Markov chains to approximate conditional distributions, with the Metropolis-within-Gibbs algorithm standing out as a well-known example. Despite their widespread use in both statistical and non-statistical applications, little is known about their convergence properties. This article introduces novel methods for establishing bounds on the convergence rates of certain reversible hybrid Gibbs samplers. In particular, we examine the convergence characteristics of hybrid random-scan Gibbs algorithms. Our analysis reveals that the absolute spectral gap of a hybrid Gibbs chain can be bounded based on the absolute spectral gap of the exact Gibbs chain and the absolute spectral gaps of the Markov chains employed for conditional distribution approximations. We also provide a convergence bound of similar flavors for hybrid data augmentation algorithms, extending existing works on the topic. The general bounds are applied to three examples: a random-scan Metropolis-within-Gibbs sampler, random-scan Gibbs samplers with block updates, and a hybrid slice sampler.

Spectral gap bounds for reversible hybrid Gibbs chains

TL;DR

This work provides a rigorous framework to bound the convergence rate of reversible hybrid Gibbs chains by relating the spectral gap of the hybrid kernel to that of the exact Gibbs chain and the quality of the conditional samplers used for approximation. Using Dirichlet forms and Markov decomposition, the authors derive general inequalities showing that the hybrid chain’s spectral gap lies within a factor governed by the approximation kernels Q_{i,y}, with tighter bounds when these kernels are PSD. The theory is illustrated through three settings: random-walk Metropolis within Gibbs, blocked coordinate updates, and a hybrid slice sampler, yielding explicit bounds on the ratio of asymptotic variances and guidance on when hybrid schemes are nearly as efficient as exact Gibbs. These results offer practical insights for implementing hybrid Gibbs methods in high-dimensional problems and provide a pathway to quantify trade-offs between computational cost and convergence speed. Overall, the paper advances understanding of hybrid MCMC convergence by linking spectral properties to the quality of conditional approximations and by delivering concrete, applicable bounds.

Abstract

Hybrid Gibbs samplers represent a prominent class of approximated Gibbs algorithms that utilize Markov chains to approximate conditional distributions, with the Metropolis-within-Gibbs algorithm standing out as a well-known example. Despite their widespread use in both statistical and non-statistical applications, little is known about their convergence properties. This article introduces novel methods for establishing bounds on the convergence rates of certain reversible hybrid Gibbs samplers. In particular, we examine the convergence characteristics of hybrid random-scan Gibbs algorithms. Our analysis reveals that the absolute spectral gap of a hybrid Gibbs chain can be bounded based on the absolute spectral gap of the exact Gibbs chain and the absolute spectral gaps of the Markov chains employed for conditional distribution approximations. We also provide a convergence bound of similar flavors for hybrid data augmentation algorithms, extending existing works on the topic. The general bounds are applied to three examples: a random-scan Metropolis-within-Gibbs sampler, random-scan Gibbs samplers with block updates, and a hybrid slice sampler.
Paper Structure (20 sections, 27 theorems, 118 equations)

This paper contains 20 sections, 27 theorems, 118 equations.

Table of Contents

  1. Introduction
  2. Overview
  3. Gibbs and hybrid Gibbs samplers
  4. Summary of results
  5. Preliminaries
  6. Random-scan Gibbs chain
  7. Data augmentation chain
  8. Examples
  9. Random-walk Metropolis within Gibbs
  10. Bayesian regression with a spike-and-slab prior
  11. A generic setting
  12. Random-scan Gibbs samplers with blocked updates
  13. A hybrid slice sampler
  14. Discussion
  15. A Technical Lemma
  16. ...and 5 more sections

Key Result

Theorem 2

Suppose that there exist constants $c_1, c_2 \in [0,2]$ such that for $i=1,\dots,n$, $M_{-i}$-a.e. $y \in \mathsf{X}_{-i}$, and $g \in L_0^2(\Pi_{i,y}) \setminus \{0\}$. Then, for $f \in L_0^2(\Pi)$,

Theorems & Definitions (44)

  • Remark 1
  • Theorem 2
  • proof
  • Remark 3
  • Corollary 4
  • proof
  • Corollary 5
  • proof
  • Lemma 6
  • Proposition 7
  • ...and 34 more