First Steps Towards a Runtime Analysis When Starting With a Good Solution

TL;DR

This paper introduces fixed-start runtime analysis for evolutionary algorithms, revealing that starting from a good initial solution can markedly alter performance and that different algorithms benefit differently from such starts. By examining the OneMax problem and the (1 + (λ, λ)) GA under static, fitness-dependent, self-adjusting, and heavy-tailed parameterizations, it derives tight upper bounds on runtime and discusses corresponding black-box complexity bounds. Key contributions include explicit bounds showing strong gains for certain algorithms when initialized near the optimum, parameterless variants with provable performance, and a formal Black-box Complexity framework for starts with distance D. Experimental results corroborate the theory, highlighting the practical value of adapting parameters to initial quality and suggesting room for discovering new EAs that exploit good initial solutions in broader problem classes.

Abstract

The mathematical runtime analysis of evolutionary algorithms traditionally regards the time an algorithm needs to find a solution of a certain quality when initialized with a random population. In practical applications it may be possible to guess solutions that are better than random ones. We start a mathematical runtime analysis for such situations. We observe that different algorithms profit to a very different degree from a better initialization. We also show that the optimal parameterization of the algorithm can depend strongly on the quality of the initial solutions. To overcome this difficulty, self-adjusting and randomized heavy-tailed parameter choices can be profitable. Finally, we observe a larger gap between the performance of the best evolutionary algorithm we found and the corresponding black-box complexity. This could suggest that evolutionary algorithms better exploiting good initial solutions are still to be found. These first findings stem from analyzing the performance of the $(1+1)$ evolutionary algorithm and the static, self-adjusting, and heavy-tailed $(1 + (λ,λ))$ GA on the OneMax benchmark. We are optimistic that the question how to profit from good initial solutions is interesting beyond these first examples.

Figures (7)

• Figure 1: Illustration of the bounding integrals for an increasing $f(x)$. In both plots the gray area is equal to the estimated sum. In the left plot the red area is an upper bound for the sum. In the right plot the blue area is the lower bound for the sum.
• Figure 2: Illustration of the bounding integrals for an decreasing $f(x)$. In both plots the gray area is equal to the estimated sum. In the left plot the red area is an upper bound for the sum. In the right plot the blue area is the lower bound for the sum.
• Figure 3: The illustration of the bounding integrals for a function which is increasing in interval $[a,c]$ and decreasing in interval $[c, b]$. In both plots the gray area is equal to the estimated sum. In the left plot the red area is an upper bound for the sum. The left part of the red line represents $f(x)$ in interval $[a, c]$, the central part shows the line $y = f(c)$ in interval $[c, c + 1]$ and the right part stands for $f(x - 1)$ in interval $[c + 1, b + 1]$. In the right plot the solid blue line stands for $\min\{f(x), f(x - 1)\}$ and the sum of blue areas is not a lower bound for the sum, unless we subtract the overlapped area in interval $[c, c + 1]$ together with the small areas above the solid blue line. This area is at most $f(c)$.
• Figure 4: Mean runtimes and their standard deviation of different algorithms on OneMax with initial Hamming distance $D$ from the optimum equal to $\sqrt{n}$ in expectation. By $\lambda \in [1..u]$ we denote the self-adjusting parameter choice via the one-fifth rule in the interval $[1..u]$. The indicated confidence interval for each value $X$ is $[E[X] - \sigma(X), E[x] + \sigma(X)]$, where $\sigma(X)$ is the standard deviation of $X$. The runtime is normalized by $\sqrt{nD}$, so that the plot of the self-adjusting ${(1 + (\lambda , \lambda))}$ GA is a horizontal line.
• Figure 6: Mean runtimes and their standard deviation of different algorithms on OneMax with problem size $n=2^{22}$ and with initial Hamming distances of the form $D = 2^i$ for $0 \le i \le 21$. The starred versions of the fast ${(1 + (\lambda , \lambda))}$ GA have a distribution upper bound of $\sqrt{n}$.

Theorems & Definitions (29)

• Lemma 1
• Lemma 2: Lemma 2.2 in AntipovBD22
• proof : Proof of Lemma \ref{['lem:progress']}
• Lemma 3
• proof
• Lemma 4
• proof
• Lemma 5
• proof
• Lemma 6: Wald's equation