Faithful global convergence for the rescaled Consensus-Based Optimization

TL;DR

This work analyzes the Consensus-Based Optimization algorithm with a consensus point rescaled by a small fixed parameter to establish its unconditional convergence to the global minimizer, providing a rigorous theoretical foundation for the algorithm's global convergence.

Abstract

We analyze the Consensus-Based Optimization (CBO) algorithm with a consensus point rescaled by a small fixed parameter $κ\in (0,1)$. Under minimal assumptions on the objective function and the initial data, we establish its unconditional convergence to the global minimizer. Our results hold in the asymptotic regime where both the time--horizon $t \to \infty$ and the inverse--temperature $α\to \infty$, providing a rigorous theoretical foundation for the algorithm's global convergence. Furthermore, our findings extend to the case of multiple and non--discrete set of minimizers.

Figures (3)

• Figure 1: We apply the standard CBO ($\kappa=1$ and $\delta=0$) and the rescaled CBO ($\kappa=0.01$ and $\delta=5$) particle system \ref{['CBOkappa particle']} to the Rastrigin function $R(x):=10+(x-1)^2-10\cos(2\pi(x-1))$, which has a unique global minimizer $x_*=1$ (the red star). The initial particles (the blue dots) are sampled uniformly in $[2,5]$ (it does not contain $x_*$). The simulation parameters are $N=100,\lambda=1,\sigma=0.5,\alpha=10^{15},{\rm d} t=0.01$ and $T=100$. The final output is $\mathfrak{m}_{\alpha}(\rho_{T}^N)$ (the green circle). Left: The standard CBO collapses prematurely to a local minimizer. Right: The rescaled CBO finds the global minimizer.
• Figure 2: Numerical test on the objective function $f(x,y)$, which has two global minimizers at $(1,1)$ and $(-1,-1)$. Left: The particles are initialized uniformly in $[-2,-1]\times [3,4]$, a region distant from the global minimizers. The CBO method successfully converges to the global minimizer $(1,1)$. Right: The particles are initialized uniformly in $[-3,-2]\times [1,2]$. The CBO method successfully converges to the global minimizer $(-1,-1)$.
• Figure 3: We test the Ackley function in dimension $d = 20$. Left: The particles are initialized uniformly in $[2,3]^{20}$ (which excludes$x_*$). The rescaled CBO ($\kappa = 0.9$, $\delta = 0.8$) converges to the global minimizer, while the standard CBO stalls at a local minimum. Right: The particles are drawn from $\mathcal{N}((2,\dots,2)^\top, \mathds{I}_{20})$ (whose support includes$x_*$). The rescaled CBO (with $\kappa = 0.9$ and $\delta = 0.8$) still outperforms the standard CBO.

Theorems & Definitions (39)

• Remark 2.2
• Lemma 2.3
• Remark 2.4
• Lemma 2.5
• proof
• Theorem 2.6
• proof
• Remark 2.7
• Lemma 2.8
• Theorem 3.1