Tail Bounds on the Runtime of Categorical Compact Genetic Algorithm

TL;DR

This paper develops a theory for the runtime of a categorical compact genetic algorithm (ccGA) with sample size two, extending the binary cGA to the categorical domain. It establishes tight tail bounds for runtimes on two representative linear objectives, COM and KVal, under explicit learning-rate regimes that depend on the problem size $D$ and the number of categories $K$, and introduces two novel drift-analytic tools: conditional drift theorems and a drift theorem for skipping processes. The main results show that the runtime on COM is, with high probability, $O(rac{\sqrt{D}\ln(DK)}{η})$ and lower-bounded by $Ω(rac{\sqrt{D}+\ln K}{η})$, while the runtime on KVal is $Θ(\frac{D \ln K}{η})$ under suitable $η$, highlighting how the category count $K$ influences search efficiency differently across linear functions. The paper also provides general drift-theorem extensions, connects to the information-geometric optimization framework, and supports the theory with experiments, offering guidance on choosing learning rates for categorical optimization problems. The results advance understanding of EDAs on categorical domains and inform practical parameter settings for scalable discrete optimization tasks.

Abstract

The majority of theoretical analyses of evolutionary algorithms in the discrete domain focus on binary optimization algorithms, even though black-box optimization on the categorical domain has a lot of practical applications. In this paper, we consider a probabilistic model-based algorithm using the family of categorical distributions as its underlying distribution and set the sample size as two. We term this specific algorithm the categorical compact genetic algorithm (ccGA). The ccGA can be considered as an extension of the compact genetic algorithm (cGA), which is an efficient binary optimization algorithm. We theoretically analyze the dependency of the number of possible categories $K$, the number of dimensions $D$, and the learning rate $η$ on the runtime. We investigate the tail bound of the runtime on two typical linear functions on the categorical domain: categorical OneMax (COM) and KVal. We derive that the runtimes on COM and KVal are $O(\sqrt{D} \ln (DK) / η)$ and $Θ(D \ln K/ η)$ with high probability, respectively. Our analysis is a generalization for that of the cGA on the binary domain.

Figures (5)

• Figure 1: The values of $1 - \theta^{(t)}_{d,1}$ (left) and $X^{(t)}_d$ defined in \ref{['eq:onemax-potential']} (right) in one typical trial of optimizing COM with $D=64$ and $K=2$. The values for each dimension are plotted as separate lines.
• Figure 2: The values of $1 - \theta^{(t)}_{d,1}$ (left) and $X^{(t)}_d$ defined in \ref{['eq:onemax-potential']} (right) in one typical trial of optimizing COM with $D=64$ and $K=20$. The values for each dimension are plotted as separate lines.
• Figure 3: The negative potential values used in the analysis of cGA Domino:2018, $- \sum_{d=1}^D (1 - \theta^{(t)}_{d,1})$ (left), and our potential values $X^{(t)}$ in \ref{['eq:kval-potential']} (right) in one typical trial of optimizing KVal with $D=16$ and $K=2$.
• Figure 4: The negative potential values used in the analysis of cGA Domino:2018, $- \sum_{d=1}^D (1 - \theta^{(t)}_{d,1})$ (left), and our potential values $X^{(t)}$ in \ref{['eq:kval-potential']} (right) in one typical trial of optimizing KVal with $D=16$ and $K=20$.
• Figure 5: The boxplots indicate experimental runtimes of ccGA on COM (left) and KVal (right). There are one hundred runs at each point. The upper bounds on COM and KVal are plotted as $D K (\ln (DK))^2$ and $D^2 K^2 \ln K \ln (DK)$ by lines without boxes, respectively.

Theorems & Definitions (37)

• Lemma 1
• Lemma 2
• Theorem 1
• Theorem 2
• Theorem 3
• Theorem 4
• Lemma 3
• proof : Proof of Lemma \ref{['thm:Delta_bound']}
• Theorem 5
• Lemma 4