Fast Conditional Mixing of MCMC Algorithms for Non-log-concave Distributions

TL;DR

This work addresses slow global mixing of MCMC for non-log-concave targets by introducing conditional mixing on subsets where local Poincaré-type inequalities hold. It develops a framework of Local Logarithmic Sobolev Inequality and Local Poincaré Inequality to establish polynomial-time conditional convergence for Langevin Monte Carlo and Gibbs sampling, with concrete rate statements and discretization-aware analyses. The authors apply these results to Gaussian mixtures and power posteriors, showing that benign local structure enables fast sampling within modes even when global mixing is slow, and they validate the theory with experiments, including restart strategies to enhance practical performance. The findings expand the toolbox for non-convex sampling by proving that efficient local sampling can be achieved under verifiable local geometric/functional-analytic conditions, with implications for mixtures, high-dimensional inference, and symmetry-rich problems.

Abstract

MCMC algorithms offer empirically efficient tools for sampling from a target distribution $π(x) \propto \exp(-V(x))$. However, on the theory side, MCMC algorithms suffer from slow mixing rate when $π(x)$ is non-log-concave. Our work examines this gap and shows that when Poincaré-style inequality holds on a subset $\mathcal{X}$ of the state space, the conditional distribution of MCMC iterates over $\mathcal{X}$ mixes fast to the true conditional distribution. This fast mixing guarantee can hold in cases when global mixing is provably slow. We formalize the statement and quantify the conditional mixing rate. We further show that conditional mixing can have interesting implications for sampling from mixtures of Gaussians, parameter estimation for Gaussian mixture models and Gibbs-sampling with well-connected local minima.

Figures (3)

• Figure 1: we plot the sampling distributions after $T$ iterations and the KL divergences w.r.t $t$
• Figure 2: An illustration for the definitions of $\tau(x)$and $\tau(x, r)$.
• Figure :

Theorems & Definitions (41)

• Definition 3.1: Conditional mixing
• Definition 4.1: Poincaré Inequality
• Definition 4.2: Logarithmic Sobolev Inequality
• Proposition 4.1: Theorem 2, balasubramanian2022towards
• Definition 4.3: Local Logarithmic Sobolev Inequality
• Lemma 4.1
• Corollary 4.1
• Definition 4.4: Local Poincaré Inequality
• Lemma 4.2
• Proposition 4.2