Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Kinetic based optimization enhanced by genetic dynamics

Giacomo Albi, Federica Ferrarese, Claudia Totzeck

TL;DR

This work addresses global optimization for non-convex high-dimensional objectives by extending kinetic-based optimization (KBO) with genetic dynamics, yielding a two-species swarm (leaders and followers) and a mean-field Boltzmann framework. GKBO combines binary interactions with GA-inspired selection to drive convergence toward the global minimizer, using a Laplace-principle–based estimate $\,hat{x}(t)$ to steer leaders. Theoretical results show exponential decay of variance and convergence to a global minimum under suitable conditions, with anisotropic diffusion aiding dimension-independent behavior. Numerical experiments on translated Rastrigin demonstrate that GKBO reduces iteration counts and improves efficiency compared to KBO and GA variants, particularly when evaluations are costly.

Abstract

We propose and analyse a variant of the recently introduced kinetic based optimization method that incorporates ideas like survival-of-the-fittest and mutation strategies well-known from genetic algorithms. Thus, we provide a first attempt to reach out from the class of consensus/kinetic-based algorithms towards genetic metaheuristics. Different generations of genetic algorithms are represented via two species identified with different labels, binary interactions are prescribed on the particle level and then we derive a mean-field approximation in order to analyse the method in terms of convergence. Numerical results underline the feasibility of the approach and show in particular that the genetic dynamics allows to improve the efficiency, of this class of global optimization methods in terms of computational cost.

Kinetic based optimization enhanced by genetic dynamics

TL;DR

This work addresses global optimization for non-convex high-dimensional objectives by extending kinetic-based optimization (KBO) with genetic dynamics, yielding a two-species swarm (leaders and followers) and a mean-field Boltzmann framework. GKBO combines binary interactions with GA-inspired selection to drive convergence toward the global minimizer, using a Laplace-principle–based estimate to steer leaders. Theoretical results show exponential decay of variance and convergence to a global minimum under suitable conditions, with anisotropic diffusion aiding dimension-independent behavior. Numerical experiments on translated Rastrigin demonstrate that GKBO reduces iteration counts and improves efficiency compared to KBO and GA variants, particularly when evaluations are costly.

Abstract

We propose and analyse a variant of the recently introduced kinetic based optimization method that incorporates ideas like survival-of-the-fittest and mutation strategies well-known from genetic algorithms. Thus, we provide a first attempt to reach out from the class of consensus/kinetic-based algorithms towards genetic metaheuristics. Different generations of genetic algorithms are represented via two species identified with different labels, binary interactions are prescribed on the particle level and then we derive a mean-field approximation in order to analyse the method in terms of convergence. Numerical results underline the feasibility of the approach and show in particular that the genetic dynamics allows to improve the efficiency, of this class of global optimization methods in terms of computational cost.
Paper Structure (18 sections, 3 theorems, 107 equations, 9 figures, 1 table)

This paper contains 18 sections, 3 theorems, 107 equations, 9 figures, 1 table.

Table of Contents

  1. Introduction
  2. Genetic kinetic based optimization (GKBO)
  3. Binary interaction between agents
  4. Emergence of leaders and followers
  5. Derivation of the mean-field equation
  6. Moments estimates and convergence to the global minimum
  7. Evolution of the moment estimates
  8. Convergence to the global minimum
  9. Numerical methods
  10. Validation tests
  11. Test 1: Comparison of different followers / leaders ratios
  12. Test 2: GKBO for different choices of $\hat{x}$.
  13. Test 3: Comparison in $d=20$ dimensions for varying $\sigma_F$
  14. Test 4: Comparison of different leader emergence strategies.
  15. Test 5: Comparison of different methods for varying $d$
  16. ...and 3 more sections

Key Result

Proposition 1

Let us assume the transitions have equilibrated, that is, $\rho_0 \equiv \rho_0^{\infty}$ and $\rho_1 \equiv \rho_1^{\infty}$. Furthermore let $\mathcal{E}(x)$ positive and bounded for all $x\in\mathbb{R}^d$, in particular, there exist constants $\underline{\mathcal{E}},\overline{\mathcal{E}} >0$ su and define $\tilde{\sigma} = k \sigma_F^2 b_{\underline{\mathcal{E}}}$, with $b_{\underline{\mathca

Figures (9)

  • Figure 1: Diagram describing the relation between the KBO, the GKBO with weighted and random strategies and the GA.
  • Figure 2: Success rates for varying $\sigma_F$ and $d$ for the translated Rastrigin function with dynamics simulated with the GKBO method for $\hat{x}$ (left), $\hat{x}_F$ (middle), $\hat{x}_L$ (right). The first row is with random leader emergence, second row with mixed strategy $\bar{p} = 0.5$ and third row with weighted leader emergence.
  • Figure 3: Success rates and means of the number of iterations as $\sigma_F$ varies and $d=20$ for the translated Rastrigin function obtained with the different algorithms. On the left leaders emerge randomly, in the, we consider mixed leader emergence with $\bar{p}=0.5$, and on the right, we have weighted leader emergence. The markers denote the value of the success rates and numbers of iterations for different $\sigma_F$.
  • Figure 4: Different leader emergence strategies. On the left, success rates for varying $\sigma_F$ and $\bar{p}$ and $d=20$. On the right, max, min and mean number of iterations obtained in the different simulations for $d =20$ and $\sigma_F=4,5$ as $\bar{p}$ varies. The markers denote the number of iterations needed for different $\bar{p}$ and tested on the translated Rastrigin function.
  • Figure 5: Different leader emergence strategies. Success rates and mean number of iterations for $d=20$ as $\sigma_F$ varies and for different values of $\bar{p}$, tested on the translated Rastigin function. The markers denote the value of the success rates and number of iterations for different $\sigma_F$.
  • ...and 4 more figures

Theorems & Definitions (12)

  • Remark 1
  • Remark 2
  • Remark 3
  • Remark 4
  • Remark 5
  • Proposition 1
  • proof
  • Corollary 1
  • proof
  • Proposition 2
  • ...and 2 more