Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rebalancing Markov jump processes for non-reversible continuous-time sampling

Erik Jansson, Moritz Schauer, Ruben Seyer, Akash Sharma

TL;DR

The paper addresses slow mixing in reversible sampling by introducing a rebalancing device that converts skew-detailed-balanced Markov jump processes into non-reversible continuous-time samplers targeting $\pi$, with minimal rejection and no repeated proposals. It builds lifted samplers (FFF and BJS) via a reference measure, a $\pi$-isometric involution $\mathfrak{s}$, and a balancing function $g$, enabling unbounded balancing and simple kernel composition. Theoretical contributions include invariance, non-explosion, and geometric ergodicity results under mild tail and smoothness conditions, along with a detailed drift–minorization analysis for the FFF class. Empirical results on Bayesian logistic regression, Rosenbrock banana, and Bayesian PKPD demonstrate robustness to tuning and improved efficiency relative to Metropolis-based and standard HMC approaches, highlighting practical scalability for Bayesian inference in complex geometries.

Abstract

Markov chain Monte Carlo methods are central in computational statistics, and typically rely on detailed balance to ensure invariance with respect to a target distribution. Although straightforward to construct by Metropolization, this can induce diffusion-like exploration of the sample space, requiring careful tuning of parameters such as step size. We introduce a general mechanism for constructing non-reversible continuous-time samplers, without requiring detailed balance. Our approach transforms jump processes satisfying a skew-detailed balance condition for a reference measure into processes sampling a target measure absolutely continuous with respect to it. Unbounded balancing functions allow such samplers to dynamically select favourable transitions. We establish invariance under weak criteria and demonstrate how to verify geometric ergodicity. Numerical experiments demonstrate that the resulting samplers are more robust to parameter tuning.

Rebalancing Markov jump processes for non-reversible continuous-time sampling

TL;DR

The paper addresses slow mixing in reversible sampling by introducing a rebalancing device that converts skew-detailed-balanced Markov jump processes into non-reversible continuous-time samplers targeting , with minimal rejection and no repeated proposals. It builds lifted samplers (FFF and BJS) via a reference measure, a -isometric involution , and a balancing function , enabling unbounded balancing and simple kernel composition. Theoretical contributions include invariance, non-explosion, and geometric ergodicity results under mild tail and smoothness conditions, along with a detailed drift–minorization analysis for the FFF class. Empirical results on Bayesian logistic regression, Rosenbrock banana, and Bayesian PKPD demonstrate robustness to tuning and improved efficiency relative to Metropolis-based and standard HMC approaches, highlighting practical scalability for Bayesian inference in complex geometries.

Abstract

Markov chain Monte Carlo methods are central in computational statistics, and typically rely on detailed balance to ensure invariance with respect to a target distribution. Although straightforward to construct by Metropolization, this can induce diffusion-like exploration of the sample space, requiring careful tuning of parameters such as step size. We introduce a general mechanism for constructing non-reversible continuous-time samplers, without requiring detailed balance. Our approach transforms jump processes satisfying a skew-detailed balance condition for a reference measure into processes sampling a target measure absolutely continuous with respect to it. Unbounded balancing functions allow such samplers to dynamically select favourable transitions. We establish invariance under weak criteria and demonstrate how to verify geometric ergodicity. Numerical experiments demonstrate that the resulting samplers are more robust to parameter tuning.

Paper Structure

This paper contains 20 sections, 13 theorems, 100 equations, 13 figures, 1 table.

Table of Contents

  1. Introduction
  2. Rebalancing device
  3. Markov jump processes
  4. Skew-detailed balance and rebalancing
  5. Construction of rate kernels and compositionality
  6. Implementation
  7. Creating samplers using the rebalancing device
  8. Hamiltonian dynamics
  9. Bouncy Particle dynamics
  10. Numerical experiments
  11. Bayesian logistic regression
  12. Rosenbrock banana
  13. Bayesian PKPD model
  14. Rebalancing device: Proofs of \ref{['sec:rebalancing']}
  15. Geometric ergodicity of FFF: Proof of \ref{['thm:geometricergodicity']}
  16. ...and 5 more sections

Key Result

Theorem 1

Let $\pi$ be a probability measure, ${\mathfrak{s}}$ be a $\pi$-isometric involution, and $\mu$ be a rate kernel with finite expected jump rate $\int_{S \times S} \pi(\mathrm{d} a)\mu(a, \mathrm{d} b) < \infty$. If a weaker form of eq:skew-db holds, for all bounded measurable $f\colon S \to \mathbb{R}$, and the semi-local condition holds with respect to ${\mathfrak{s}}$ and $\pi$, then there exi

Figures (13)

  • Figure 1: Suboptimal hyperparameters hinders the reversible HMC over short trajectories. Non-reversible methods are more robust, and by reducing rejections, our methods (FFF and BJS) explore further for the same computational cost.
  • Figure 2: Minimum marginal ESS per $10^3$ gradient evaluations (higher is better) on Bayesian logistic regression target for varying hyperparameters, averaged across replicates.
  • Figure 3: Comparison of minimum marginal ESS per $10^3$ gradient evaluations (higher is better) and proportion of flips (lower is better) on Bayesian logistic regression target for varying $\varepsilon$ with best $L$ and $\lambda_\rho$, averaged across replicates. Error bars show $\pm 2$ standard errors of the mean.
  • Figure 4: Minimum marginal ESS per wall clock second (higher is better) on Bayesian logistic regression target for varying hyperparameters, averaged across replicates.
  • Figure 5: Comparison of minimum marginal ESS per wall clock second (higher is better) and per $10^3$ gradient evaluations (higher is better) on Bayesian logistic regression target for varying $\varepsilon$ with best $\lambda_\rho$, averaged across replicates. Error bars show $\pm 2$ standard errors of the mean.
  • ...and 8 more figures

Theorems & Definitions (34)

  • Definition 1
  • Remark 1
  • Theorem 1
  • Theorem 2
  • Remark 2
  • Remark 3
  • Remark 4
  • Proposition 1
  • Proposition 2
  • proof
  • ...and 24 more