Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evolving Benchmark Functions to Compare Evolutionary Algorithms via Genetic Programming

Yifan He, Claus Aranha

TL;DR

This work tackles the challenge of automatically generating benchmark functions that differentiate between evolutionary optimizers. It leverages Genetic Programming to evolve functions, using the Wasserstein distance between the optimizers' solution distributions as the fitness objective and MAP-Elites to maintain landscape-diverse benchmarks guided by fitness-landscape descriptors. The approach is validated against the CEC2005 benchmark across case studies involving differential evolution parametrizations and strong optimizers (SHADE and CMA-ES), showing improved differentiation in the decision space and competitive differentiation in the objective space. The method offers a principled, automated tool for benchmarking and analyzing algorithm behavior, with potential extensions to fitness-space distances and richer landscape descriptors for hyper-heuristic integration.

Abstract

In this study, we use Genetic Programming (GP) to compose new optimization benchmark functions. Optimization benchmarks have the important role of showing the differences between evolutionary algorithms, making it possible for further analysis and comparisons. We show that the benchmarks generated by GP are able to differentiate algorithms better than human-made benchmark functions. The fitness measure of the GP is the Wasserstein distance of the solutions found by a pair of optimizers. Additionally, we use MAP-Elites to both enhance the search power of the GP and also illustrate how the difference between optimizers changes by various landscape features. Our approach provides a novel way to automate the design of benchmark functions and to compare evolutionary algorithms.

Evolving Benchmark Functions to Compare Evolutionary Algorithms via Genetic Programming

TL;DR

This work tackles the challenge of automatically generating benchmark functions that differentiate between evolutionary optimizers. It leverages Genetic Programming to evolve functions, using the Wasserstein distance between the optimizers' solution distributions as the fitness objective and MAP-Elites to maintain landscape-diverse benchmarks guided by fitness-landscape descriptors. The approach is validated against the CEC2005 benchmark across case studies involving differential evolution parametrizations and strong optimizers (SHADE and CMA-ES), showing improved differentiation in the decision space and competitive differentiation in the objective space. The method offers a principled, automated tool for benchmarking and analyzing algorithm behavior, with potential extensions to fitness-space distances and richer landscape descriptors for hyper-heuristic integration.

Abstract

In this study, we use Genetic Programming (GP) to compose new optimization benchmark functions. Optimization benchmarks have the important role of showing the differences between evolutionary algorithms, making it possible for further analysis and comparisons. We show that the benchmarks generated by GP are able to differentiate algorithms better than human-made benchmark functions. The fitness measure of the GP is the Wasserstein distance of the solutions found by a pair of optimizers. Additionally, we use MAP-Elites to both enhance the search power of the GP and also illustrate how the difference between optimizers changes by various landscape features. Our approach provides a novel way to automate the design of benchmark functions and to compare evolutionary algorithms.
Paper Structure (20 sections, 7 equations, 7 figures, 5 tables, 3 algorithms)

This paper contains 20 sections, 7 equations, 7 figures, 5 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Optimization Benchmarks
  4. Techniques Used
  5. Koza's Genetic Programming
  6. MAP-Elites
  7. Fitness Landscape Analysis
  8. Proposed Method
  9. Function Set and Terminals
  10. Evaluation Metrics
  11. MAP-Elites using Landscape Metrics
  12. Experiments
  13. Case Study on Parametrization of Differential Evolution
  14. Training performance
  15. Test performance
  16. ...and 5 more sections

Figures (7)

  • Figure 1: General outline of the proposed GP system
  • Figure 2: Phenotypic descriptor of the MAP-Elites in our study
  • Figure 3: Average Wasserstein distance of the functions in the archive in case study 1
  • Figure 4: Average Wasserstein distance of the functions in the archive in case study 2
  • Figure 5: Functions with Top-5 Wasserstein distance in the two case studies (first line: case study 1, second line: case study 2)
  • ...and 2 more figures