A localized consensus-based sampling algorithm

TL;DR

This work develops localized consensus-based sampling (localized CBS), a gradient-free, affine-invariant sampler for non-Gaussian targets that arises from connecting interacting Langevin dynamics to consensus-based sampling via a Moreau envelope, a proximal-operator approximation, and a Gaussian-assumption reduction. It retains full gradient-free operation and parallelizability while delivering improved robustness over polarized CBS in non-Gaussian settings; in the mean-field limit, it is exact for Gaussian targets. The paper provides mean-field analyses for Gaussian targets, discusses generalized preconditioners, and demonstrates favorable empirical performance on Gaussian, multimodal, and inverse problems such as Darcy flow, highlighting its potential for challenging Bayesian inverse problems with unknown normalizing constants. The methodology offers a scalable alternative to gradient-based samplers and complements existing ensemble methods by enabling localized interactions and flexible preconditioning. Overall, localized CBS broadens the toolkit for gradient-free Bayesian sampling in high-dimensional, non-Gaussian contexts with promising theoretical and practical implications.

Abstract

We propose a localized consensus-based method for sampling from non-Gaussian distributions. This method arises from an alternative derivation of consensus-based sampling (CBS). Starting from ensemble-preconditioned Langevin dynamics, we approximate the potential with a Moreau envelope, replace the gradient in the Langevin equation with a proximal operator, and finally approximate this operator by a weighted mean. Under Gaussian initial and target distributions, this procedure recovers the standard CBS dynamics. In addition, when we retain only the approximations valid beyond the Gaussian case, we retrieve a refined variant of polarized CBS. The resulting algorithm, which we call localized consensus-based sampling, is affine-invariant, exact for Gaussian targets in the mean-field limit, and demonstrates improved robustness over polarized CBS in numerical experiments. Like other consensus-based methods, localized CBS is fully gradient-free and easily parallelizable.

Figures (7)

• Figure 1: Localized CBS samples for a one-dimensional Gaussian target distribution
• Figure 2: Marginalized samples for a 1- or 10-dimensional multimodal target distribution. Recall that we aggregate the samples from 16 runs, which explains the non-Gaussian CBS samples.
• Figure 3: Marginalized samples for the poorly scaled distribution with potential \ref{['eq:num:multi:V-ai']}. In the left figure, the two red lines are visually indistinguishable; in the middle figure, one line coincides with the horizontal axis.
• Figure 4: Localized CBS samples for \ref{['eq:num:multi:diffpeaks']} with various preconditioners and initial distributions
• Figure 5: Localized and polarized CBS samples for \ref{['eq:num:multi:diffpeaks']}

Theorems & Definitions (18)

• Remark 1.1: Another concept of localization
• Definition 2.1: Affine-invariance
• Lemma 4.1
• proof
• Corollary 4.2
• proof
• Remark 4.3: Non-Gaussian performance
• Lemma 4.4
• proof
• Lemma 5.1