Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Runtime Analysis for Multi-Objective Evolutionary Algorithms in Unbounded Integer Spaces

Benjamin Doerr, Martin S. Krejca, Günter Rudolph

TL;DR

This work initiates a theoretical runtime analysis of multi-objective evolutionary algorithms in unbounded integer spaces by studying SEMO and GSEMO under three mutation laws on Rudolph's benchmark, where the Pareto front size scales as $|F^*|=2a+1$. It reveals a nuanced trade-off: unit-step mutation progresses slowly when far from the front, exponential-tail mutation can be fast given a carefully chosen $q$ but is highly sensitive to problem parameters, and power-law mutation provides robust, parameter-light performance across scenarios. Theoretical results are complemented by empirical experiments showing power-law mutation often outperforms even a well-tuned exponential-tail variant, and in many cases yields near-linear runtimes in problem size. The findings advocate using power-law mutation for unknown integer-space MO problems and lay the groundwork for further exploration of population dynamics and lower bounds in (G)SEMO-type algorithms.

Abstract

Randomized search heuristics have been applied successfully to a plethora of problems. This success is complemented by a large body of theoretical results. Unfortunately, the vast majority of these results regard problems with binary or continuous decision variables -- the theoretical analysis of randomized search heuristics for unbounded integer domains is almost nonexistent. To resolve this shortcoming, we start the runtime analysis of multi-objective evolutionary algorithms, which are among the most successful randomized search heuristics, for unbounded integer search spaces. We analyze single- and full-dimensional mutation operators with three different mutation strengths, namely changes by plus/minus one (unit strength), random changes following a law with exponential tails, and random changes following a power-law. The performance guarantees we prove on a recently proposed natural benchmark problem suggest that unit mutation strengths can be slow when the initial solutions are far from the Pareto front. When setting the expected change right (depending on the benchmark parameter and the distance of the initial solutions), the mutation strength with exponential tails yields the best runtime guarantees in our results -- however, with a wrong choice of this expectation, the performance guarantees quickly become highly uninteresting. With power-law mutation, which is an essentially parameter-less mutation operator, we obtain good results uniformly over all problem parameters and starting points. We complement our mathematical findings with experimental results that suggest that our bounds are not always tight. Most prominently, our experiments indicate that power-law mutation outperforms the one with exponential tails even when the latter uses a near-optimal parametrization. Hence, we suggest to favor power-law mutation for unknown problems in integer spaces.

Runtime Analysis for Multi-Objective Evolutionary Algorithms in Unbounded Integer Spaces

TL;DR

This work initiates a theoretical runtime analysis of multi-objective evolutionary algorithms in unbounded integer spaces by studying SEMO and GSEMO under three mutation laws on Rudolph's benchmark, where the Pareto front size scales as . It reveals a nuanced trade-off: unit-step mutation progresses slowly when far from the front, exponential-tail mutation can be fast given a carefully chosen but is highly sensitive to problem parameters, and power-law mutation provides robust, parameter-light performance across scenarios. Theoretical results are complemented by empirical experiments showing power-law mutation often outperforms even a well-tuned exponential-tail variant, and in many cases yields near-linear runtimes in problem size. The findings advocate using power-law mutation for unknown integer-space MO problems and lay the groundwork for further exploration of population dynamics and lower bounds in (G)SEMO-type algorithms.

Abstract

Randomized search heuristics have been applied successfully to a plethora of problems. This success is complemented by a large body of theoretical results. Unfortunately, the vast majority of these results regard problems with binary or continuous decision variables -- the theoretical analysis of randomized search heuristics for unbounded integer domains is almost nonexistent. To resolve this shortcoming, we start the runtime analysis of multi-objective evolutionary algorithms, which are among the most successful randomized search heuristics, for unbounded integer search spaces. We analyze single- and full-dimensional mutation operators with three different mutation strengths, namely changes by plus/minus one (unit strength), random changes following a law with exponential tails, and random changes following a power-law. The performance guarantees we prove on a recently proposed natural benchmark problem suggest that unit mutation strengths can be slow when the initial solutions are far from the Pareto front. When setting the expected change right (depending on the benchmark parameter and the distance of the initial solutions), the mutation strength with exponential tails yields the best runtime guarantees in our results -- however, with a wrong choice of this expectation, the performance guarantees quickly become highly uninteresting. With power-law mutation, which is an essentially parameter-less mutation operator, we obtain good results uniformly over all problem parameters and starting points. We complement our mathematical findings with experimental results that suggest that our bounds are not always tight. Most prominently, our experiments indicate that power-law mutation outperforms the one with exponential tails even when the latter uses a near-optimal parametrization. Hence, we suggest to favor power-law mutation for unknown problems in integer spaces.

Paper Structure

This paper contains 21 sections, 20 theorems, 38 equations, 2 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Algorithmic Framework
  4. Mutation.
  5. Runtime.
  6. Benchmark Problem
  7. Useful Properties of the Benchmark Problem
  8. Optimization Dynamics
  9. Runtime Analysis
  10. Mathematical Tools
  11. Unit-Step Mutation
  12. Exponential-Tail Mutation
  13. Power-Law Mutation
  14. Empirical Analysis
  15. Theoretical considerations.
  16. ...and 6 more sections

Key Result

Lemma 1

Let $x \in \mathbb{Z}^n$ and $y \in \mathbb{Z}_{\geq a} \times \mathbb{Z}^{n - 1}$ such that $f_1(x) \leq f_1(y)$. Then $f(x) \preceq f(y)$. Similarly, let $x \in \mathbb{Z}^n$ and $y \in \mathbb{Z}_{\leq -a} \times \mathbb{Z}^{n - 1}$ such that $f_2(x) \leq f_2(y)$. Then $f(x) \preceq f(y)$.

Figures (2)

  • Figure 1: The results of scenario $2$. Average evaluations of $f$ for varying $a$ for the GSEMO optimizing $f$ with the mutation operators: unit-step (diamonds), exponential-tail (cross diamonds) with $\frac{1}{q} = \frac{a}{4}$, and power-law (triangles) with $\beta = \frac{3}{2}$. Each point is based on $50$ independent runs, with $x^{(0)} = (0, 100 a)$. The dotted lines depict the std. deviations.
  • Figure 2: The results from scenario $2$ for $n = 4$ (left) and $n = 10$ (right). Average function evaluations for varying $a$ for the GSEMO optimizing $f$ with three different mutation operators: unit-step (brown diamonds), exponential-tail (blue diamonds) with $\frac{1}{q} = \frac{a}{4}$, and power-law (red triangles) with $\beta = \frac{3}{2}$. For each point, $50$ independent runs were conducted, choosing $x^{(0)}_2 = 100a$ and all other components of $x^{(0)}$ as $0$. The dotted lines depict the standard deviation of each curve.

Theorems & Definitions (37)

  • Lemma 1
  • Lemma 2
  • Lemma 3
  • Lemma 4
  • Lemma 5
  • Lemma 6
  • Theorem 7: Discrete variable drift, upper bound
  • Theorem 8: Additive drift, unbounded search space
  • Theorem 9
  • Lemma 10
  • ...and 27 more