Sub-Cauchy Sampling: Escaping the Dark Side of the Moon

TL;DR

The paper addresses the challenge of sampling from heavy-tailed posteriors in Bayesian models, where standard Metropolis–Hastings methods often fail to be geometrically or uniformly ergodic. It introduces Sub-Cauchy Projection (SCP), a geometric transform that maps $\mathbb{R}^d$ onto a spherical cap, and a Metropolis sampler on the bright side called the Sub-Cauchy Projection Sampler (SCS). The authors prove uniform ergodicity of SCS for sub-Cauchy targets, develop a variational approach to tune the projection parameters, and demonstrate strong performance on high-dimensional heavy-tailed problems, including robust binary regression and multivariate skew-$t$ distributions, where competing methods struggle. Collectively, SCP/SCS provide a broadly applicable, provably convergent tool for Bayesian inference with heavy tails, delivering improved tail exploration and computational efficiency over existing techniques such as SPS, RWM, and HMC in the tested scenarios.

Abstract

We introduce a Markov chain Monte Carlo algorithm based on Sub-Cauchy Projection, a geometric transformation that generalizes stereographic projection by mapping Euclidean space into a spherical cap of a hyper-sphere, referred to as the complement of the dark side of the moon. We prove that our proposed method is uniformly ergodic for sub-Cauchy targets, namely targets whose tails are at most as heavy as a multidimensional Cauchy distribution, and show empirically its performance for challenging high-dimensional problems. The simplicity and broad applicability of our approach open new opportunities for Bayesian modeling and computation with heavy-tailed distributions in settings where most existing methods are unreliable.

Figures (9)

• Figure 1: Illustration of a 2-dimensional Sub-Cauchy Projection with $o = (0, 0, 1.5)$ (top panels) and $o = (0.6, 0.6, 1.5)$ (bottom panels). Right panels: contours of log density of the uniform distribution on the bright side of the sphere projected onto $\mathbb{R}^2$.
• Figure 2: Sampling from a 100-dimensional standard Cauchy distribution. Left panel: trace plot of $\|X\|^2$ of SCS (orange), HMC (blue), SPS (green), and RWM (yellow), initializing from a sample drawn from the target. Middle panel: trajectory in the first two dimensions (on the log scale) of SCS (orange), HMC (blue), and RWM (yellow), initializing from $(10^3, 10^3,\ldots)$. Right panel: density of the SCS samples of $\|X\|^2$ (red, estimated by kernel density regression), compared against the true density (blue).
• Figure 3: SP (top panels) and SCP with observer $o=(0,\, 0,\, 1.1)$ (bottom panels) of 2-dimensional Cauchy distribution. Right panels correspond to the unnormalized density evaluated along the latitude component $\ell_x$. The density on the sphere peaks to infinity on the north pole for SP, while it is bounded from above and below for SCP.
• Figure 4: Top view of the stepping-out mechanism. The shaded area represents the dark side. Starting from $x$, the point $x'$ was proposed in the dark side and the stepping-out function bring it back to the bright-side $x^\star$.
• Figure 5: Q--Q plot of SCS (red) and HMC (blue) in sampling a 100-dimensional multivariate skew $t$ distribution with 2 (top) or 1 (bottom) degrees of freedom. Shaded regions show variation across 10 independent replicates.

Theorems & Definitions (15)

• Definition 1
• Proposition 1
• Example 1
• Example 2
• Remark 1
• Proposition 2
• Theorem 1
• Corollary 1: Uniform ergodicity of SCS for sub-Cauchy distributions
• proof
• Remark 2