Ballistic Convergence in Hit-and-Run Monte Carlo and a Coordinate-free Randomized Kaczmarz Algorithm

TL;DR

This work quantifies the advantages of coordinate-free Monte Carlo and optimization methods by establishing sharp Wasserstein contraction bounds for generalized Hit-and-Run targeting Gaussian measures. A unifying contraction lemma for random projections enables precise rates $\rho=\tfrac{1}{2}\inf_{|\zeta|=1} \mathbb{E}_{v\sim\tau} (\zeta\cdot\frac{\mathcal{C}^{-1/2}v}{|\mathcal{C}^{-1/2}v|})^2$, which drive fast mixing and reveal ballistic or superdiffusive behavior in certain low/high-mode regimes. The analysis extends naturally to a coordinate-free randomized Kaczmarz algorithm, yielding analogous convergence rates in terms of $A^Tv$ and enabling potential speed-ups over the classical, coordinate-restricted variant. Together, these results provide rigorous justification for the advantages and limitations of coordinate-free methods in both sampling and optimization, with explicit dependence on target geometry and dimension.

Abstract

Hit-and-Run is a coordinate-free Gibbs sampler, yet the quantitative advantages of its coordinate-free property remain largely unexplored beyond empirical studies. In this paper, we prove sharp estimates for the Wasserstein contraction of Hit-and-Run in Gaussian target measures via coupling methods and conclude mixing time bounds. Our results uncover ballistic and superdiffusive convergence rates in certain settings. Furthermore, we extend these insights to a coordinate-free variant of the randomized Kaczmarz algorithm, an iterative method for linear systems, and demonstrate analogous convergence rates. These findings offer new insights into the advantages and limitations of coordinate-free methods for both sampling and optimization.

Figures (20)

• Figure 1: Comparison of 10 steps of Gibbs vs. Hit-and-Run in a narrow bivariate Gaussian target with condition number $\kappa\gg1$: Gibbs is limited to small, incremental steps along the coordinate axes, resulting in a slow diffusive mixing rate of $\kappa^{-1}$. In contrast, Hit-and-Run samples directions uniformly and can take large, global steps whenever its sampled direction sufficiently aligns with the major axis of the ellipse. This enables Hit-and-Run to achieve a faster ballistic mixing rate of $\kappa^{-1/2}$ (see Section \ref{['sec:ballistic']}).
• Figure 3: Transition step of Hit-and-Run from current state $x$ to next state $X$.
• Figure 4: Two realizations of a Hit-and-Run transition starting from $x$ with $\mathcal{C}=\operatorname{diag}(\kappa,1)$, $\kappa>1$, see \ref{['eq:HRstep']}. The directions $v_{\mathrm{local}}$ and $v_{\mathrm{global}}$ result in vastly different moves. The distribution $\mathrm{Law}(\mathcal{C}^{-1/2}v/|\mathcal{C}^{-1/2}v|)$, for $v\sim\mathrm{Unif}(\mathbb S^1)$, favors directions that lead to local moves. However, the probability of making a global move is of order $\kappa^{-1/2}$, see Figure \ref{['fig:wlaw']}, allowing Hit-and-Run to achieve ballistic mixing, as discussed in Section \ref{['sec:ballistic']}.
• Figure 6: This figure shows contour lines of the transition density of the Hit-and-Run kernel in \ref{['eq:xyHRkernel']} starting from two different points: $x=(-2,0)$ (black) and $\tilde{x}=(0,1)$ (gray). The covariance matrix is $\mathcal{C}=\operatorname{diag}(4,1)$. A contour line of the target density (dotted) is included for comparison.
• Figure : (a) Difference between components of a synchronous coupling of two copies of Hit-and-Run Monte Carlo.

Theorems & Definitions (16)

• Lemma 1
• proof : Proof of Lemma \ref{['lem:contrproj']}
• Theorem 1
• proof
• Lemma 2
• proof : Proof of Lemma \ref{['lem:HRcontr']}
• Lemma 3
• proof
• Theorem 2
• Lemma 4