From Consensus-Based Optimization to Evolution Strategies: Proof of Global Convergence
Massimo Fornasier, Hui Huang, Jona Klemenc, Greta Malaspina
TL;DR
This work establishes rigorous global convergence guarantees for a family of zero-order global optimization methods by tying Consensus-Based Optimization (CBO) to MPPI and Evolution Strategies (ES) through mean-field and nonlinear Fokker–Planck analyses. It introduces two new variants, δ-CBO and Consensus Freezing, and derives a time-rescaled Consensus Hopping scheme that corresponds to MPPI/ES, all possessing explicit invariant Gaussian measures and exponential convergence rates. The authors develop a fixed-point framework anchored in a refined Laplace principle and optimal-transport tools, yielding exact invariant measures and quantitative convergence in Wasserstein distance, even in the infinite-time horizon and large-α limits. They further provide stable numerical implementations and demonstrate robustness to large time steps, along with a precise link between CF and CH as the resampling speed grows. The results illuminate how to systematically bridge CBO with ES/MPPI while maintaining provable global convergence, offering practical guidance for scalable, parallelizable optimization in nonconvex, nonsmooth landscapes.
Abstract
Consensus-based optimization (CBO) is a powerful and versatile zero-order multi-particle method designed to provably solve high-dimensional global optimization problems, including those that are genuinely nonconvex or nonsmooth. The method relies on a balance between stochastic exploration and contraction toward a consensus point, which is defined via the Laplace principle as a proxy for the global minimizer. In this paper, we introduce new CBO variants that address practical and theoretical limitations of the original formulation of this novel optimization methodology. First, we propose a model called $δ$-CBO}, which incorporates nonvanishing diffusion to prevent premature collapse to suboptimal states. We also develop a numerically stable implementation, the Consensus Freezing scheme, that remains robust even for arbitrarily large time steps by freezing the consensus point over time intervals. We connect these models through appropriate asymptotic limits. Furthermore, we derive from the Consensus Freezing scheme by suitable time rescaling and asymptotics a further algorithm, the Consensus Hopping scheme, which can be interpreted as a form of $(1,λ)$-Evolution Strategy. For all these schemes, we characterize for the first time the invariant measures and establish global convergence results, including exponential convergence rates.