Positive autocorrelation at unit lag for stationary random walk Metropolis-Hastings in ${\mathbb R}^d$
James Allen Fill, Svante Janson
TL;DR
This work shows that stationary random-walk Metropolis–Hastings chains in $\mathbb{R}^d$ exhibit positive unit-lag autocorrelation for nonzero linear projections, by bounding the incremental $r$th-absolute-moment of $X_1-X_0$ and relating it to the variance structure of $X_0$. It provides general moment formulas for MH and RWMH (continuous and integer-grid), yields sharp lower bounds under unimodal and spherically symmetric targets, and extends the results to linear functionals and higher dimensions. The paper also analyzes the impact of symmetrizing the target or the proposal, derives analogous inequalities for IMH, and proves nonnegative even-lag autocovariances for reversible MH chains. Collectively, these results clarify the dependence structure and offer rigorous, dimension-aware guarantees on unit-lag and even-lag correlations that inform practical MH behavior and proposal design.
Abstract
It is often asserted in the literature that one should expect positive autocorrelation for random walk Metropolis-Hastings (RWMH), especially if the typical proposal step-size is small relative to the variability in the target density. In this paper, we consider a stationary RWMH chain ${\bf X}$ taking values in $d$-dimensional Euclidean space and (subject only to the existence of densities with respect to Lebesgue measure) with general target distribution having finite second moment and general proposal random walk step-distribution. We prove, for any nonzero vector ${\bf c}$, strict positivity of the autocorrelation function at unit lag for the stochastic process $\langle{\bf c},{\bf X}\rangle$, that is, \[{\operatorname{Corr}}(\langle{\bf c},{\bf X}_0\rangle,\langle{\bf c},{\bf X}_1\rangle)>0,\] and we establish the same result, but with weak inequality (which can in some cases be equality) when the state space for ${\bf X}$ is changed to the integer grid ${\mathbb Z}^d$. Further, for ${\bf c}\neq{\bf 0}$ we establish the sharp lower bound \[{\operatorname{Corr}}(\langle{\bf c},{\bf X}_0\rangle,\langle{\bf c},{\bf X}_1\rangle)>\tfrac19\] on autocorrelation when we assume both that (i) the target density $π$ is spherically symmetric and unimodal in the specific sense that $π({\bf x})=\hatπ(\|{\bf x}\|)$ for some nonincreasing function $\hatπ$ on $[0,\infty)$ and that (ii) the proposal step-density is symmetric about ${\bf 0}$. We study the autocorrelation indirectly, by considering the incremental variance function (or incremental second-moment function) at unit lag. The same approach allows us also for $r\in[2,\infty)$ to upper-bound the incremental $r$th-absolute-moment function at unit lag. We give also closely related inequalities for the total variation distance between two distributions on ${\mathbb R}^d$ differing only by a location shift.