Almanac: HMC sampling with bounded velocity

TL;DR

This work assesses Hamiltonian Monte Carlo samplers with bounded-velocity kinetic energies for cosmological sky-map/posterior problems that exhibit funnel-like non-Gaussian regions. By evaluating relativistic and Student's $t$ kinetic energies within Almanac, and comparing Classic versus Cholesky parameterizations, the study quantifies effects on convergence (FMI), mixing (correlation length $\hat{\tau}$), and sampling efficiency (ESR) across mid- and high-dimensional problems. Key findings indicate that the Cholesky parameterization markedly improves convergence and efficiency, heavy-tailed momentum can boost mixing at the cost of speed, and moderately bounded velocities (e.g., Relativistic-$2M$) often offer the best balance, though gains are problem-dependent and diminish with dimensionality. The results underscore the importance of sampler design and point toward adaptive, self-tuning approaches to momentum and parameterization in high-dimensional HMC applications.

Abstract

In Hamiltonian Monte Carlo sampling, the shape of the potential and the choice of the momentum distribution jointly give rise to the Hamiltonian dynamics of the sampler. An efficient sampler propagates quickly in all regions of the parameter space, so that the chain has a low autocorrelation length and the sampler has a high acceptance rate, with the goal of optimising the number of near-independent samples for given computational cost. Standard Gaussian momentum distributions allow arbitrarily large velocities, which can lead to inefficient exploration in posteriors with ridges or funnel-like geometries. We investigate alternative momentum distributions based on relativistic and Student's t kinetic energies, which naturally limit particle velocities and may improve robustness. Using Almanac, a sampler for cosmological posterior distributions of sky maps and power spectra on the sphere, we test these alternatives in both low- and high-dimensional settings. We find that the choice of parameterization and momentum distribution can improve convergence and effective sample rate, though the achievable gains are generally modest and strongly problem-dependent, reaching up to an order of magnitude in favorable cases. Among the momentum distributions that we tested, those with moderately heavy tails achieved the best balance between efficiency and stability. These results highlight the importance of sampler design and encourage future work on adaptive and self-tuning strategies for kinetic energy parameter optimization in high-dimensional settings.

Figures (5)

• Figure 1: Momentum proposal probability density functions in one dimension: Relativistic (orange and green dashed line), Student's t (pink and gray dots), and Gaussian (blue solid line). The non-Gaussian distributions have fatter tails (and hence propose large momenta more often); apart from that the distributions appear similar. This visual similarity masks how strongly the non-Gaussian distributions change the Hamiltonian flow: their gradients ensure that particles cannot propagate above a maximum velocity (see Fig. \ref{['dist_v']}). In each case we show the right half of a density function that is symmetric about $p=0$. The standard deviations of the plotted distributions are: $\sqrt{2}$ for the Relativistic($c=1$, $m=0$) and for the $\nu=4$ Student's t, unity for the Relativistic($c=2$, $m=0.597$) and for the Gaussian, and undefined for the $\nu=2$ Student's t.
• Figure 2: Velocity $\dot{q} = p/M(p)$ as a function of $p$ for the same examples as in Fig. \ref{['dist_p']}. The Gaussian distribution (with quadratic kinetic energy) has unconstrained velocity (blue solid line). For the relativistic kinetic energy, we have a maximum velocity of $c$ for a photon (orange solid line) and a massive particle (green dashed line). The mass determines the particle's behaviour near the minimum of the potential, and the maximum velocity in the outskirts. For the Student's t kinetic energy (pink and gray dotted lines), the maximum velocity is given by Eq. \ref{['max_velocity_student']}. For each non-Gaussian example, the $y\text{-axis}$ is normalised by the maximum velocity $\dot{q}_{\rm max}$.
• Figure 3: Velocity proposal distributions for same examples as in Figs. \ref{['dist_p']} and \ref{['p_vs_v']}. The Gaussian distribution (blue solid line) allows all velocities. For the relativistic kinetic energy, we observe a delta function at $\dot{q}=1$ when $m=0$ (orange dashed line) and for massive particles a distribution that proposes velocities in the range $(-1,1)$, which is less likely to propose velocities close to the maximum (green dashed line). In contrast, for the Student's t kinetic energy (pink and gray dotted lines), it is more likely to propose velocities close to the maximum. For each non-Gaussian example, the $x\text{-axis}$ is normalised by maximum velocity $\dot{q}_{\rm max}$. In each case we show the right half of a density function that is symmetric about $\dot{q}=0$.
• Figure 4: The Effective Sample Rate depends on the parameter considered. Here we plot the distributions of the spectrum variables for the 'high-dimensional' set of experiments in Section \ref{['sec:almanac_experiments']}. The colours and markers indicate the different kinetic energy definitions. The $x\text{-axis}$ is log-spaced and the $y\text{-axis}$ shows the probability density of this logarithmic variable. See Section \ref{['sec:results']} for a discussion of the 'two pairs' appearance of this plot.
• Figure 5: Effective Sample Rate as function of multipole moment $\ell$, ordered according to the different power spectra in the two redshift bins for the high-dim case. The colours and markers indicate the different kinetic energy definitions used in the sampler. Note how the efficiency of the different spectra changes according to $\ell$, especially those containing $B-$modes. For clarity, only a subset of $\ell$ values between 4 and 1024 is displayed, plotted every 30 modes.