Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CMA-ES with Adaptive Reevaluation for Multiplicative Noise

Kento Uchida, Kenta Nishihara, Shinichi Shirakawa

TL;DR

This work analyzes how noise affects CMA-ES and shows that the standard noise-dependent utility used in some variants can fail under multiplicative noise. It introduces RA-CMA-ES, a reevaluation-adaptive CMA-ES that computes two update directions from half of the evaluations and adjusts the number of reevaluations based on the estimated correlation of those directions, enhanced by learning-rate adaptation. Through experiments on 10-dimensional benchmarks with additive and multiplicative noise, RA-CMA-ES demonstrates superior performance under multiplicative noise and competitive results under additive noise. The approach advances robust noisy optimization by linking reevaluation effort to the reliability of update directions via correlation estimation and SNR-aware adaptation.

Abstract

The covariance matrix adaptation evolution strategy (CMA-ES) is a powerful optimization method for continuous black-box optimization problems. Several noise-handling methods have been proposed to bring out the optimization performance of the CMA-ES on noisy objective functions. The adaptations of the population size and the learning rate are two major approaches that perform well under additive Gaussian noise. The reevaluation technique is another technique that evaluates each solution multiple times. In this paper, we discuss the difference between those methods from the perspective of stochastic relaxation that considers the maximization of the expected utility function. We derive that the set of maximizers of the noise-independent utility, which is used in the reevaluation technique, certainly contains the optimal solution, while the noise-dependent utility, which is used in the population size and leaning rate adaptations, does not satisfy it under multiplicative noise. Based on the discussion, we develop the reevaluation adaptation CMA-ES (RA-CMA-ES), which computes two update directions using half of the evaluations and adapts the number of reevaluations based on the estimated correlation of those two update directions. The numerical simulation shows that the RA-CMA-ES outperforms the comparative method under multiplicative noise, maintaining competitive performance under additive noise.

CMA-ES with Adaptive Reevaluation for Multiplicative Noise

TL;DR

This work analyzes how noise affects CMA-ES and shows that the standard noise-dependent utility used in some variants can fail under multiplicative noise. It introduces RA-CMA-ES, a reevaluation-adaptive CMA-ES that computes two update directions from half of the evaluations and adjusts the number of reevaluations based on the estimated correlation of those directions, enhanced by learning-rate adaptation. Through experiments on 10-dimensional benchmarks with additive and multiplicative noise, RA-CMA-ES demonstrates superior performance under multiplicative noise and competitive results under additive noise. The approach advances robust noisy optimization by linking reevaluation effort to the reliability of update directions via correlation estimation and SNR-aware adaptation.

Abstract

The covariance matrix adaptation evolution strategy (CMA-ES) is a powerful optimization method for continuous black-box optimization problems. Several noise-handling methods have been proposed to bring out the optimization performance of the CMA-ES on noisy objective functions. The adaptations of the population size and the learning rate are two major approaches that perform well under additive Gaussian noise. The reevaluation technique is another technique that evaluates each solution multiple times. In this paper, we discuss the difference between those methods from the perspective of stochastic relaxation that considers the maximization of the expected utility function. We derive that the set of maximizers of the noise-independent utility, which is used in the reevaluation technique, certainly contains the optimal solution, while the noise-dependent utility, which is used in the population size and leaning rate adaptations, does not satisfy it under multiplicative noise. Based on the discussion, we develop the reevaluation adaptation CMA-ES (RA-CMA-ES), which computes two update directions using half of the evaluations and adapts the number of reevaluations based on the estimated correlation of those two update directions. The numerical simulation shows that the RA-CMA-ES outperforms the comparative method under multiplicative noise, maintaining competitive performance under additive noise.
Paper Structure (26 sections, 3 theorems, 55 equations, 3 figures, 1 table, 1 algorithm)

This paper contains 26 sections, 3 theorems, 55 equations, 3 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. CMA-ES
  4. Noise-Resilient Variants of CMA-ES
  5. Uncertainty Handling CMA-ES
  6. Population Size Adaptation CMA-ES
  7. Learning Rate Adaptation CMA-ES
  8. Stochastic Relaxation in Noisy Optimization
  9. Possible Setting of Utility Functions
  10. Noise-dependent utility:
  11. Noise-independent utility:
  12. Discussion on Preferable Utility Function
  13. Adaptive Reevaluation
  14. Estimation of Update Correlation
  15. Proposed Method: RA-CMA-ES
  16. ...and 11 more sections

Key Result

lemma 1

Assume the following conditions: Then, for any distribution parameter $\boldsymbol{\theta} \in \Theta$, there exist a function $f$ and a noise distribution $P_{n}$ that hold where $\boldsymbol{x}^\ast$ is an arbitrary optimal solution in eq:original-problem.

Figures (3)

  • Figure 1: Result with multiplicative Gaussian noise on 10-dimensional benchmark problems
  • Figure 2: Result with multiplicative uniform noise on 10-dimensional benchmark problems
  • Figure 3: Result with additive Gaussian noise on 10-dimensional benchmark problems

Theorems & Definitions (7)

  • lemma 1
  • Remark 1
  • lemma 2
  • lemma 3
  • proof
  • proof
  • proof