Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Superlinearly Convergent Evolution Strategy

Tobias Glasmachers

TL;DR

This paper introduces the Quasi-Newton Evolution Strategy (QN-ES), a hybrid optimizer that replaces the ES recombination update with a quasi-Newton mean step $p = -B^{-1} g$ while leveraging Hessian-estimation from HE-ES to adapt curvature. A gradient estimator derived from mirrored offspring samples yields an approximate gradient $\nabla f(Ax) \approx \delta$, with $\nabla f(x) \approx A^{-1} \delta$, producing the Newton-like update $p = -\eta A^T \delta$ where $\eta$ scales curvature; the method also uses a switching mechanism to balance recombination and Newton steps. Empirical results show superlinear convergence on smooth convex problems and clear performance gains in late convergence phases compared to HE-ES, though robustness can be challenged on highly non-convex landscapes and under multi-modal conditions where restarts influence outcomes. Overall, QN-ES advances ES by integrating a gradient-informed, quasi-Newton step, offering faster convergence on suitable problems while suggesting a pathway toward tighter ES-DFO integration and theoretical convergence analysis.

Abstract

We present a hybrid algorithm between an evolution strategy and a quasi Newton method. The design is based on the Hessian Estimation Evolution Strategy, which iteratively estimates the inverse square root of the Hessian matrix of the problem. This is akin to a quasi-Newton method and corresponding derivative-free trust-region algorithms like NEWUOA. The proposed method therefore replaces the global recombination step commonly found in non-elitist evolution strategies with a quasi-Newton step. Numerical results show superlinear convergence, resulting in improved performance in particular on smooth convex problems.

A Superlinearly Convergent Evolution Strategy

TL;DR

This paper introduces the Quasi-Newton Evolution Strategy (QN-ES), a hybrid optimizer that replaces the ES recombination update with a quasi-Newton mean step while leveraging Hessian-estimation from HE-ES to adapt curvature. A gradient estimator derived from mirrored offspring samples yields an approximate gradient , with , producing the Newton-like update where scales curvature; the method also uses a switching mechanism to balance recombination and Newton steps. Empirical results show superlinear convergence on smooth convex problems and clear performance gains in late convergence phases compared to HE-ES, though robustness can be challenged on highly non-convex landscapes and under multi-modal conditions where restarts influence outcomes. Overall, QN-ES advances ES by integrating a gradient-informed, quasi-Newton step, offering faster convergence on suitable problems while suggesting a pathway toward tighter ES-DFO integration and theoretical convergence analysis.

Abstract

We present a hybrid algorithm between an evolution strategy and a quasi Newton method. The design is based on the Hessian Estimation Evolution Strategy, which iteratively estimates the inverse square root of the Hessian matrix of the problem. This is akin to a quasi-Newton method and corresponding derivative-free trust-region algorithms like NEWUOA. The proposed method therefore replaces the global recombination step commonly found in non-elitist evolution strategies with a quasi-Newton step. Numerical results show superlinear convergence, resulting in improved performance in particular on smooth convex problems.
Paper Structure (13 sections, 5 equations, 5 figures, 1 table, 2 algorithms)

This paper contains 13 sections, 5 equations, 5 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. The Hessian Estimation Evolution Strategy
  4. Quasi-Newton Methods
  5. The Quasi Newton Evolution Strategy
  6. Gradient and Quasi-Newton Step
  7. Switch between Recombination and Quasi-Newton
  8. Where to place QN-ES as a Hybrid?
  9. Experimental Evaluation
  10. Convergence Speed
  11. Challenges
  12. Black-Box Optimization Benchmarking
  13. Discussion and Conclusion

Figures (5)

  • Figure 1: Convergence plots (best-so-far $f(x) - f(x^*)$ over the number of function evaluations) of Powell's method (green stars), HE-ES (blue circles), and QN-ES (orange diamonds) for the problems sphere, ellipsoid, discus, cigar, and Rosenbrock (top to bottom) in dimensions 5 (left) and 20 (right). The plots show typical single runs. In order to assess convergence behavior the problems are solved to a high precision of $f(x) - f(x^*) \leq 10^{-20}$.
  • Figure 2: Multiplicative progress of the function value per iteration of QN-ES when minimizing a 10-dimensional Rosenbrock function to very high precision. Note the logarithmic scale of the vertical axis. After iteration 170, superlinear convergence (a steady increase of the convergence rate) is clearly visible. For this experiment, the optimum of the Rosenbrock function was shifted to the origin in order to make optimal use of floating point precision.
  • Figure 3: Convergence plots of Powell's method (green stars), HE-ES (blue dots), and QN-ES (orange diamonds) for the functions log-sphere, sum of different powers, one-norm, and happycat (top to bottom) in dimensions 5 (left) and 20 (right). The plots show single runs.
  • Figure 4: COCO empirical cumulative distribution function (ECDF) plots by BBOB function group in problem dimension 5. The plots show the fraction of reached targets over the number of function evaluations divided by dimension (higher/more to the left is better). Note that beyond the large cross, the curves are extrapolated (based on "virtual restarts").
  • Figure 5: COCO empirical cumulative distribution function (ECDF) plots by BBOB function group in problem dimension 20. The plots show the fraction of reached targets over the number of function evaluations divided by dimension (higher/more to the left is better). Note that beyond the large cross, the curves are extrapolated (based on "virtual restarts").