Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Score-based Metropolis-Hastings for Fractional Langevin Algorithms

Ahmed Aloui, Junyi Liao, Ali Hasan, Jose Blanchet, Vahid Tarokh

TL;DR

The paper addresses the challenge of sampling from heavy-tailed, multimodal targets when both the target density and the $\alpha$-stable proposal density are intractable for traditional Metropolis–Hastings corrections. It introduces MAFLA, a Metropolis-adjusted, score-based correction that operates without density evaluations by enforcing a gradient form of detailed balance via Score Balance Matching and learning an acceptance function from target-score and proposal-score proxies derived from isotropic $\alpha$-stable structure. The approach combines a density-free approximation of the fractional proposal drift with a trained acceptance mechanism, yielding improved finite-time sampling on heavy-tailed targets and better exploration-stability trade-offs in combinatorial optimization relaxations (e.g., MaxCut, Vertex Cover). Empirical results show substantial gains in tail accuracy and mixture weight recovery compared to unadjusted FULA, as well as competitive performance against traditional MALA/ULA baselines on graph-based optimization tasks. This framework advances Metropolis-type corrections for Lévy-driven samplers and demonstrates practical benefits for challenging, high-dimensional, heavy-tailed problems.

Abstract

Sampling from heavy-tailed and multimodal distributions is challenging when neither the target density nor the proposal density can be evaluated, as in $α$-stable Lévy-driven fractional Langevin algorithms. While the target distribution can be estimated from data via score-based or energy-based models, the $α$-stable proposal density and its score are generally unavailable, rendering classical density-based Metropolis--Hastings (MH) corrections impractical. Consequently, existing fractional Langevin methods operate in an unadjusted regime and can exhibit substantial finite-time errors and poor empirical control of tail behavior. We introduce the Metropolis-Adjusted Fractional Langevin Algorithm (MAFLA), an MH-inspired, fully score-based correction mechanism. MAFLA employs designed proxies for fractional proposal score gradients under isotropic symmetric $α$-stable noise and learns an acceptance function via Score Balance Matching. We empirically illustrate the strong performance of MAFLA on a series of tasks including combinatorial optimization problems where the method significantly improves finite time sampling accuracy over unadjusted fractional Langevin dynamics.

Score-based Metropolis-Hastings for Fractional Langevin Algorithms

TL;DR

The paper addresses the challenge of sampling from heavy-tailed, multimodal targets when both the target density and the -stable proposal density are intractable for traditional Metropolis–Hastings corrections. It introduces MAFLA, a Metropolis-adjusted, score-based correction that operates without density evaluations by enforcing a gradient form of detailed balance via Score Balance Matching and learning an acceptance function from target-score and proposal-score proxies derived from isotropic -stable structure. The approach combines a density-free approximation of the fractional proposal drift with a trained acceptance mechanism, yielding improved finite-time sampling on heavy-tailed targets and better exploration-stability trade-offs in combinatorial optimization relaxations (e.g., MaxCut, Vertex Cover). Empirical results show substantial gains in tail accuracy and mixture weight recovery compared to unadjusted FULA, as well as competitive performance against traditional MALA/ULA baselines on graph-based optimization tasks. This framework advances Metropolis-type corrections for Lévy-driven samplers and demonstrates practical benefits for challenging, high-dimensional, heavy-tailed problems.

Abstract

Sampling from heavy-tailed and multimodal distributions is challenging when neither the target density nor the proposal density can be evaluated, as in -stable Lévy-driven fractional Langevin algorithms. While the target distribution can be estimated from data via score-based or energy-based models, the -stable proposal density and its score are generally unavailable, rendering classical density-based Metropolis--Hastings (MH) corrections impractical. Consequently, existing fractional Langevin methods operate in an unadjusted regime and can exhibit substantial finite-time errors and poor empirical control of tail behavior. We introduce the Metropolis-Adjusted Fractional Langevin Algorithm (MAFLA), an MH-inspired, fully score-based correction mechanism. MAFLA employs designed proxies for fractional proposal score gradients under isotropic symmetric -stable noise and learns an acceptance function via Score Balance Matching. We empirically illustrate the strong performance of MAFLA on a series of tasks including combinatorial optimization problems where the method significantly improves finite time sampling accuracy over unadjusted fractional Langevin dynamics.
Paper Structure (35 sections, 2 theorems, 66 equations, 9 figures, 2 tables, 2 algorithms)

This paper contains 35 sections, 2 theorems, 66 equations, 9 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background
  4. Fractional Langevin Algorithm
  5. Score Matching
  6. Metropolis-Hastings
  7. Approximating Proposal Scores
  8. Score Balance Matching
  9. Entropy regularization.
  10. Metropolis-Adjusted Fractional Langevin
  11. Experiments
  12. Conclusion
  13. Appendix
  14. MAFLA
  15. Score Balance Matching
  16. ...and 20 more sections

Key Result

Lemma 1

Let $\mathcal{X}\subseteq\mathbb{R}^d$ and assume that the target density $p:\mathcal{X}\to\mathbb{R}_{+}$, the proposal density $q(\cdot\mid x):\mathcal{X}\to\mathbb{R}_{+}$, and the acceptance function $a:\mathcal{X}\times\mathcal{X}\to(0,1]$ are differentiable. Then the following statements are e where $\nabla=(\nabla_x,\nabla_{x'})$.

Figures (9)

  • Figure 1: Illustration of symmetric $\alpha$-stable distributions for different tail indices $\alpha$. Smaller $\alpha$ yields heavier tails and higher peak near the origin.
  • Figure 2: Qualitative comparison on a 2D $\alpha$-stable mixture with weights $(0.2, 0.8)$ for $\alpha = 1.95$. MAFLA produces samples that visually match both modes and mixture weights more closely than FULA, which oversamples the left mode and exhibits noisier samples.
  • Figure 3: Effect of the proposal stability index. Left: Mean $W_1$ distance for FULA and MAFLA over a grid of target $\alpha_{\text{tgt}}$ and proposal $\alpha_{\text{prop}}$. Right: Best $95\%$ quantile error (mean $\pm$ std over runs) as a function of $\alpha_{\text{tgt}}$, comparing FULA (gray) and MAFLA (blue).
  • Figure 4: Quantile error ratio. MAFLA uniformly dominates as the ratio is greater than $1$.
  • Figure 5: Sensitivity to step size $\tau$ for FULA and MAFLA in a $4$-D $\alpha$-stable mixture, with $\alpha=1.5$. Left:$99\%$ quantile error. Center:$95\%$ quantile error. Right: Wasserstein distance $W_1$. MAFLA exhibits lower errors across all $\tau$ and improves tail behavior.
  • ...and 4 more figures

Theorems & Definitions (6)

  • Definition 1: Fisher Divergence
  • Definition 2: Sliced Score Matching
  • Definition 3: Denoising Score Matching
  • Lemma 1: Gradient Detailed Balance Equivalence
  • Lemma 2
  • proof