A Smoothing Consensus-Based Optimization Algorithm for Nonsmooth Nonconvex Optimization
Jiazhen Wei, Wei Bian
TL;DR
This paper addresses the global optimization of a possibly nonsmooth nonconvex objective $f:\mathbb{R}^d\to\mathbb{R}_+$ by introducing the Smoothing Consensus-Based Optimization (SCBO) algorithm, a finite-particle variant of consensus-based optimization that uses a shared environmental noise. It replaces $f$ by a smoothing $\tilde{f}(x,\mu)$ with a decreasing smoothing parameter $\mu(t)$ and proves global consensus and almost-sure convergence to a common state under mild parameter conditions, independent of dimension. An error bound is derived showing the consensus achieved by the smoothing-based state yields ${\rm ess}\inf f(x_\infty)\le f_{min}+E(\beta)$ with $E(\beta)\to0$ as $\beta\to\infty$, while a suitable choice of parameters guarantees small $E(\beta)$. Numerical experiments on a variety of nonsmooth, nonconvex tests confirm strong performance and demonstrate favorable comparisons with existing CBO variants, highlighting the method's robustness and dimension-independence.
Abstract
Lately, a novel swarm intelligence model, namely the consensus-based optimization (CBO) algorithm, was introduced to deal with the global optimization problems. Limited by the conditions of Ito's formula, the convergence analysis of the previous CBO finite particle system mainly focuses on the problem with smooth objective function. With the help of smoothing method, this paper achieves a breakthrough by proposing an effective CBO algorithm for solving the global solution of a nonconvex, nonsmooth, and possible non-Lipschitz continuous minimization problem with theoretical analysis, which dose not rely on the mean-field limit. We indicate that the proposed algorithm exhibits a global consensus and converges to a common state with any initial data. Then, we give a more detailed error estimation on the objective function values along the state of the proposed algorithm towards the global minimum. Finally, some numerical examples are presented to illustrate the appreciable performance of the proposed method on solving the nonsmooth, nonconvex minimization problems.