Self-Repellent Random Walks on General Graphs -- Achieving Minimal Sampling Variance via Nonlinear Markov Chains

TL;DR

This work introduces Self-Repellent Random Walks (SRRWs) on general graphs, a class of nonlinear Markov chains whose transition probabilities depend on the walker’s entire visitation history through a polynomial repellence function $r_{\mu_i}(x_i) = \left(\frac{x_i}{\mu_i}\right)^{-\alpha}$. The authors prove that the occupation measure converges almost surely to the target distribution $\boldsymbol{\mu}$ for all $\alpha\ge 0$, and establish a central limit theorem with an explicit asymptotic covariance $\mathbf{V}(\alpha)$ that decreases with increasing $\alpha$ in the Loewner order, yielding reduced sampling variance. They derive $\mathbf{V}(\alpha)$ in closed form via the spectrum of the base chain $\mathbf{P}$ and show an $O(1/\alpha)$ rate for variance reduction in MCMC settings, with numerical simulations validating the theory and illustrating the benefits of time-varying $\alpha$ to combine fast mixing with low asymptotic variance. The results suggest nonlinear Markov kernels as a principled route to more efficient sampling on graphs, with practical implications for MCMC sampling, network analysis, and distributed inference.

Abstract

We consider random walks on discrete state spaces, such as general undirected graphs, where the random walkers are designed to approximate a target quantity over the network topology via sampling and neighborhood exploration in the form of Markov chain Monte Carlo (MCMC) procedures. Given any Markov chain corresponding to a target probability distribution, we design a self-repellent random walk (SRRW) which is less likely to transition to nodes that were highly visited in the past, and more likely to transition to seldom visited nodes. For a class of SRRWs parameterized by a positive real α, we prove that the empirical distribution of the process converges almost surely to the the target (stationary) distribution of the underlying Markov chain kernel. We then provide a central limit theorem and derive the exact form of the arising asymptotic co-variance matrix, which allows us to show that the SRRW with a stronger repellence (larger α) always achieves a smaller asymptotic covariance, in the sense of Loewner ordering of co-variance matrices. Especially for SRRW-driven MCMC algorithms, we show that the decrease in the asymptotic sampling variance is of the order O(1/α), eventually going down to zero. Finally, we provide numerical simulations complimentary to our theoretical results, also empirically testing a version of SRRW with α increasing in time to combine the benefits of smaller asymptotic variance due to large α, with empirically observed faster mixing properties of SRRW with smaller α.

Figures (2)

• Figure 1: Simulations of the SRRW process for values of $\alpha \in [0,2]$, where $\alpha = 0$ corresponds to MHRW - the underlying Metropolis-Hastings base chain, with no self-repellence properties. The two sigmoid functions refer to the case where $\alpha$ is made to gradually increase over time, from $0$ to $2$. Sigmoid-1 is of the type $\frac{1}{0.5+e^{-n+0.25 N}}$ while Sigmoid-2 is of the type $\frac{n}{100+0.5n}$, where $n\geq0$ is the time parameter. Further tuning of the sigmoid functions may lead to empirically more efficient MCMC algorithms.
• Figure 2: Simulations of the SRRW process for values of $\alpha \in [0,4]$, where $\alpha = 0$ corresponds to MHRW - the underlying Metropolis-Hastings base chain, with no self-repellence properties.

Theorems & Definitions (30)

• Proposition 2.1
• Remark 2.2
• Proposition 3.1
• Lemma 3.2
• Theorem 3.3
• Lemma 3.4
• Theorem 4.1: Almost sure convergence to target $\boldsymbol{\mu}$
• Theorem 4.2: Central Limit Theorem
• Corollary 4.3: Ordering of asymptotic covariance
• Corollary 4.4