Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Consensus-Based Optimization Beyond Finite-Time Analysis

Pascal Bianchi, Radu-Alexandru Dragomir, Victor Priser

TL;DR

The paper analyzes a Consensus-Based Optimization algorithm with fixed, small noise in the long-time regime for non-convex optimization. It develops a mean-field framework via a quantitative Laplace principle that links the consensus point to a proximal operator, yielding exponential convergence to x^* up to an O(1/√α) bias, and provides a clipped mean-field SDE to obtain uniform moment bounds. For finite numbers of particles, a block-wise, time-window analysis delivers explicit long-time error bounds for each particle and proves consistency of the global best toward x^*. The results rely on propagation of chaos, Wasserstein-2 control, and discretization error analysis to establish long-time convergence and tight bounds as n, k → ∞, with explicit dependence on α and the step sequence η_k, offering practical guidance for using CBO in long-horizon optimization.

Abstract

We analyze a zeroth-order particle algorithm for the global optimization of a non-convex function, focusing on a variant of Consensus-Based Optimization (CBO) with small but fixed noise intensity. Unlike most previous studies restricted to finite horizons, we investigate its long-time behavior with fixed parameters. In the mean-field limit, a quantitative Laplace principle shows exponential convergence to a neighborhood of the minimizer x * . For finitely many particles, a block-wise analysis yields explicit error bounds: individual particles achieve long-time consistency near x * , and the global best particle converge to x * . The proof technique combines a quantitative Laplace principle with block-wise control of Wasserstein distances, avoiding the exponential blow-up typical of Gr{ö}nwall-based estimates.

Consensus-Based Optimization Beyond Finite-Time Analysis

TL;DR

The paper analyzes a Consensus-Based Optimization algorithm with fixed, small noise in the long-time regime for non-convex optimization. It develops a mean-field framework via a quantitative Laplace principle that links the consensus point to a proximal operator, yielding exponential convergence to x^* up to an O(1/√α) bias, and provides a clipped mean-field SDE to obtain uniform moment bounds. For finite numbers of particles, a block-wise, time-window analysis delivers explicit long-time error bounds for each particle and proves consistency of the global best toward x^*. The results rely on propagation of chaos, Wasserstein-2 control, and discretization error analysis to establish long-time convergence and tight bounds as n, k → ∞, with explicit dependence on α and the step sequence η_k, offering practical guidance for using CBO in long-horizon optimization.

Abstract

We analyze a zeroth-order particle algorithm for the global optimization of a non-convex function, focusing on a variant of Consensus-Based Optimization (CBO) with small but fixed noise intensity. Unlike most previous studies restricted to finite horizons, we investigate its long-time behavior with fixed parameters. In the mean-field limit, a quantitative Laplace principle shows exponential convergence to a neighborhood of the minimizer x * . For finitely many particles, a block-wise analysis yields explicit error bounds: individual particles achieve long-time consistency near x * , and the global best particle converge to x * . The proof technique combines a quantitative Laplace principle with block-wise control of Wasserstein distances, avoiding the exponential blow-up typical of Gr{ö}nwall-based estimates.

Paper Structure

This paper contains 24 sections, 18 theorems, 98 equations, 3 tables.

Table of Contents

  1. Introduction
  2. On the proof technique
  3. Mean field limit
  4. The approach of Fornasier_2024
  5. Alternative approach
  6. Finite number of particles
  7. Consistency of the global best
  8. Main results
  9. Mean-field regime
  10. Finite particle regime
  11. Proofs
  12. Proof of Proposition \ref{['prop:McKean']}
  13. Proof of Theorem \ref{['th:convergence']}
  14. Proximal operator
  15. ODE satisfied by the expectation
  16. ...and 9 more sections

Key Result

Proposition 1

Under Assumption hyp:f-flot, for any initial distribution $\nu_0 \in {\mathcal{P}}_2(\mathbb{R}^d)$ and any $T > 0$, the SDE eq:SDE-clip admits a unique strong solution $(X_t)$ on $[0,T]$. Let $y \in {\mathbb R}^d$. Then for every $t \geq 0$, there exists a Gaussian variable $Z_t \sim \mathcal{N}(0, where $x_t$ solves the ODE: Moreover, $\|x_t\| \leq \max(\|y\|,R)$ for every $t \geq 0$.

Theorems & Definitions (36)

  • Remark 1
  • Proposition 1
  • proof
  • Theorem 1
  • proof
  • Remark 2
  • Theorem 2
  • proof
  • Remark 3
  • Theorem 3
  • ...and 26 more