Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A discrete Consensus-Based Global Optimization Method with Noisy Objective Function

Stefania Bellavia, Greta Malaspina

TL;DR

This work addresses global optimization when objective evaluations are noisy by extending a discrete-time consensus-based optimization (CBO) method to stochastic function estimates. It proves that, under suitable noise conditions, the particle system achieves almost-sure consensus and that the expected mean-squared distance to the global minimizer can be made arbitrarily small, with a quantified convergence rate and an $O(\log(\varepsilon^{-1}))$ iteration bound. A quantitative Laplace-principle argument yields an $O(1/\alpha)$ bound on the suboptimality at the consensus point, linking the noise level, diffusion, and the temperature-like parameter $\alpha$. Numerical experiments on Rastrigin with Gaussian noise and on finite-sum problems with subsampling demonstrate robustness to noise and substantial computational savings, supporting the method's practicality for noisy, large-scale global optimization.

Abstract

Consensus based optimization is a derivative-free particles-based method for the solution of global optimization problems. Several versions of the method have been proposed in the literature, and different convergence results have been proved. However, all existing results assume the objective function to be evaluated exactly at each iteration of the method. In this work, we extend the convergence analysis of a discrete-time CBO method to the case where only a noisy stochastic estimator of the objective function can be computed at a given point. In particular we prove that under suitable assumptions on the oracle's noise, the expected value of the mean squared distance of the particles from the solution can be made arbitrarily small in a finite number of iterations. Numerical experiments showing the impact of noise are also given.

A discrete Consensus-Based Global Optimization Method with Noisy Objective Function

TL;DR

This work addresses global optimization when objective evaluations are noisy by extending a discrete-time consensus-based optimization (CBO) method to stochastic function estimates. It proves that, under suitable noise conditions, the particle system achieves almost-sure consensus and that the expected mean-squared distance to the global minimizer can be made arbitrarily small, with a quantified convergence rate and an iteration bound. A quantitative Laplace-principle argument yields an bound on the suboptimality at the consensus point, linking the noise level, diffusion, and the temperature-like parameter . Numerical experiments on Rastrigin with Gaussian noise and on finite-sum problems with subsampling demonstrate robustness to noise and substantial computational savings, supporting the method's practicality for noisy, large-scale global optimization.

Abstract

Consensus based optimization is a derivative-free particles-based method for the solution of global optimization problems. Several versions of the method have been proposed in the literature, and different convergence results have been proved. However, all existing results assume the objective function to be evaluated exactly at each iteration of the method. In this work, we extend the convergence analysis of a discrete-time CBO method to the case where only a noisy stochastic estimator of the objective function can be computed at a given point. In particular we prove that under suitable assumptions on the oracle's noise, the expected value of the mean squared distance of the particles from the solution can be made arbitrarily small in a finite number of iterations. Numerical experiments showing the impact of noise are also given.
Paper Structure (12 sections, 19 theorems, 149 equations, 1 figure, 6 tables)

This paper contains 12 sections, 19 theorems, 149 equations, 1 figure, 6 tables.

Table of Contents

  1. Introduction
  2. Consensus based optimization with noisy functions
  3. Convergence Analysis
  4. Diameter and Ergodicity
  5. Almost sure consensus
  6. Handling the noise
  7. Almost sure convergence
  8. Convergence in expectation
  9. Numerical Results
  10. Rastrigin Function
  11. Subsampling
  12. Conclusions

Key Result

Lemma 3.3

Let ${y}^s_{k+1}$ and $M^s_k$ be given in Y2. For any $k\in\mathbb{N}$ and $s=1,\dots,d$ we have

Figures (1)

  • Figure 1: Accuracy vs computational cost on problem \ref{['class_objfun']} on the Rice dataset for different values of $\ell$, for $N=500,$$\gamma = 0.01$ and $\xi = 0.1$

Theorems & Definitions (41)

  • Remark 3.1
  • Remark 3.2
  • Lemma 3.3
  • Lemma 3.4
  • proof
  • Theorem 3.5
  • proof
  • Theorem 3.6
  • Corollary 3.7
  • proof
  • ...and 31 more