Spectral gap of Metropolis-within-Gibbs under log-concavity
Cecilia Secchi, Giacomo Zanella
TL;DR
The paper analyzes MwG with Random Walk MH updates for $d$-dimensional log-concave targets with condition number $\kappa$. By tuning the proposal variance to the conditional variance, it derives a spectral-gap lower bound of order $O\big(1/(\kappa d)\big)$ for the random-scan MwG, improving over the prior $O\big(1/(\kappa^2 d)\big)$ bound. The main technical contribution is a sharp, dimension-aware conductance analysis in one dimension, yielding explicit lower bounds on the conductance and, via Cheeger, on the spectral gap; these extend to MwG through a coordinate-wise decomposition and yield mixing-time bounds of order $O(\kappa^* d \log(1/\varepsilon))$. The results are corroborated by a hierarchical logistic regression application, showing that variance-adaptive MwG can be as efficient as exact Gibbs sampling up to a constant factor, thereby providing theoretical support for the empirical speedups observed in practice.
Abstract
The Metropolis-within-Gibbs (MwG) algorithm is a widely used Markov Chain Monte Carlo method for sampling from high-dimensional distributions when exact conditional sampling is intractable. We study MwG with Random Walk Metropolis (RWM) updates, using proposal variances tuned to match the target's conditional variances. Assuming the target $π$ is a $d$-dimensional log-concave distribution with condition number $κ$, we establish a spectral gap lower bound of order $\mathcal{O}(1/κd)$ for the random-scan version of MwG, improving on the previously available $\mathcal{O}(1/κ^2 d)$ bound. This is obtained by developing sharp estimates of the conductance of one-dimensional RWM kernels, which can be of independent interest. The result shows that MwG can mix substantially faster with variance-adaptive proposals and that its mixing performance is just a constant factor worse than that of the exact Gibbs sampler, thus providing theoretical support to previously observed empirical behavior.