Kinetic description and convergence analysis of genetic algorithms for global optimization

TL;DR

This work presents a kinetic-theory-based framework for genetic algorithms (GA), casting GA as a stochastic particle system with binary crossover and mutation and selection as a sampling mechanism. By invoking propagation of chaos, the authors derive a mono-particle kinetic model and a Boltzmann-type equation, establishing convergence results for GA under Boltzmann selection and Gaussian mutation. They further connect GA to KBO and, in a mean-field limit, to Consensus-Based Optimization (CBO) via quasi-invariant scaling, revealing structural parallels and differences. Numerical experiments validate the kinetic approximation, reveal steady-state configurations, and illustrate the scaling interactions with CBO, demonstrating the practical relevance of the kinetic perspective for understanding and improving GA-based global optimization. The results bridge metaheuristics and kinetic theory, offering a rigorous route to analyze convergence and to relate GA to existing mean-field optimization frameworks.

Abstract

Genetic Algorithms (GA) are a class of metaheuristic global optimization methods inspired by the process of natural selection among individuals in a population. Despite their widespread use, a comprehensive theoretical analysis of these methods remains challenging due to the complexity of the heuristic mechanisms involved. In this work, relying on the tools of statistical physics, we take a first step towards a mathematical understanding of GA by showing how their behavior for a large number of individuals can be approximated through a time-discrete kinetic model. This allows us to prove the convergence of the algorithm towards a global minimum under mild assumptions on the objective function for a popular choice of selection mechanism. Furthermore, we derive a time-continuous model of GA, represented by a Boltzmann-like partial differential equation, and establish relations with other kinetic and mean-field dynamics in optimization. Numerical experiments support the validity of the proposed kinetic approximation and investigate the asymptotic configurations of the GA particle system for different selection mechanisms and benchmark problems.

Figures (4)

• Figure 1: Evolution of 1-Wasserstein distance between $f^N_{(k)}$ and a reference solution $\overline{f}_{(k)}$ obtained with $10^5$ particles and that approximates the kinetic evolution. Different population sizes $N$ and selection mechanisms are considered. Continuous lines show the mean over $100$ realizations, while shaded area indicate the [0.1,0.9] quantile interval. The objective considered is the Ackley function. Parameters are set to $\tau = 0.1, \gamma = 0.2 (1, \dots, 1)^\top$, different mutation strategies are considered, and the initial particles' locations are independently sampled from $\textup{Unif}[-2,2]$.
• Figure 2: Final particle distributions for $N = 10^6$ particles after $k_{\max} = 10^3$ iterations, for different objectives and selection mechanisms. Parameters are set to $\tau = 0.1, \sigma = 0.1, \gamma = 0.2 (1, \dots, 1)^\top$. The initial particles' locations are independently sampled from $\textup{Unif}[-2,2]$. The black and red check marks on the $y$-axis refer to the objective function and probability distribution, respectively.
• Figure 3: Error $|X^{(k)}_\textup{best} - x^\star|$ at the last iteration $k = k_{\max} = 300$ for algorithms \ref{['eq:GA2CBO']} ($\varepsilon = 1$), \ref{['eq:GA2CBOeps']} ($\varepsilon =0.3$), \ref{['eq:GA2CBOtau']} ($\varepsilon =\tau = 0.1$) and CBO. Problems' dimension is $\textup{d} =10$. Parameters are set to $\tau = 0.1, \alpha = 10^4, \lambda = 1, \sigma_{(0)} = 1$. the initial particles' locations are independently sampled from $\textup{Unif}[-2,2]^\textup{d}$. The box charts display median, lower and upper quartiles, and outliers (computed with interquartile range), for the 100 experiments performed.
• Figure 4: Accuracy $\mathcal{E}(X^{(k)}_\textup{best}) - \mathcal{E}(x^\star)$ at the last iteration $k = k_{\max} = 300$ for algorithms \ref{['eq:GA2CBO']} ($\varepsilon = 1$), \ref{['eq:GA2CBOeps']} ($\varepsilon =0.3$), \ref{['eq:GA2CBOtau']} ($\varepsilon =\tau = 0.1$) and CBO. Problems' dimension is $\textup{d} =10$. Parameters are set to $\tau = 0.1, \alpha = 10^4, \lambda = 1, \sigma_{(0)} = 1$. the initial particles' locations are independently sampled from $\textup{Unif}[-2,2]^\textup{d}$. The box charts display median, lower and upper quartiles, and outliers (computed with interquartile range), for the 100 experiments performed.

Theorems & Definitions (13)

• Remark 2.1
• Remark 3.1
• Remark 3.2
• Theorem 4.1
• Remark 4.1
• Lemma 4.1
• proof
• Lemma 4.2
• proof
• Proposition 4.1