Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Kinetic models for optimization: a unified mathematical framework for metaheuristics

Giacomo Borghi, Michael Herty, Lorenzo Pareschi

Abstract

Metaheuristic algorithms, widely used for solving complex non-convex and non-differentiable optimization problems, often lack a solid mathematical foundation. In this review, we explore how concepts and methods from kinetic theory can offer a potential unifying framework for a variety of metaheuristic optimization methods. By applying principles from collisional and non-collisional kinetic theory, we outline how particle-based algorithms like Simulated Annealing, Genetic Algorithms, Particle Swarm Optimization, and Ensemble Kalman Filter may be described through a common statistical perspective. This approach not only provides a path to deeper theoretical insights and connects different methods under suitable asymptotic scalings, but also enables the derivation of novel algorithms using alternative numerical solvers. While not exhaustive, our review highlights how kinetic models can enhance the mathematical understanding of existing optimization algorithms and inspire new computational strategies.

Kinetic models for optimization: a unified mathematical framework for metaheuristics

Abstract

Metaheuristic algorithms, widely used for solving complex non-convex and non-differentiable optimization problems, often lack a solid mathematical foundation. In this review, we explore how concepts and methods from kinetic theory can offer a potential unifying framework for a variety of metaheuristic optimization methods. By applying principles from collisional and non-collisional kinetic theory, we outline how particle-based algorithms like Simulated Annealing, Genetic Algorithms, Particle Swarm Optimization, and Ensemble Kalman Filter may be described through a common statistical perspective. This approach not only provides a path to deeper theoretical insights and connects different methods under suitable asymptotic scalings, but also enables the derivation of novel algorithms using alternative numerical solvers. While not exhaustive, our review highlights how kinetic models can enhance the mathematical understanding of existing optimization algorithms and inspire new computational strategies.

Paper Structure

This paper contains 22 sections, 5 theorems, 92 equations, 4 algorithms.

Table of Contents

  1. Introduction
  2. Simulated Annealing
  3. The heuristic strategy
  4. A linear kinetic model
  5. Convergence results
  6. Diffusion scaling
  7. Genetic Algorithm
  8. The heuristic strategy
  9. A Boltzmann-type kinetic model
  10. Discussion on propagation of chaos assumption
  11. Convergence result
  12. Quasi-invariant scaling
  13. Particle Swarm Optimization
  14. The heuristic strategy
  15. Derivation of the kinetic model
  16. ...and 7 more sections

Key Result

Theorem 2.1

If $\lambda = \lambda(t)$ satisfies eq:salog with $\lambda(t) = C_2 t^\alpha$, $C_2>0, \alpha \in (0,1)$, then for $T(t) = (C_1\log(t))^{-1}$, $C_1>0$, we have for $t\geq t_0$ sufficiently large where $\kappa = 1-\alpha$.

Theorems & Definitions (11)

  • Theorem 2.1: pareschi2024sa
  • Remark 2.1
  • Remark 3.1
  • Remark 3.2
  • Remark 3.3
  • Theorem 3.1: borghi2024ga
  • Theorem 4.1: huang2023pso
  • Remark 4.1
  • Theorem 5.1
  • Theorem 5.2
  • ...and 1 more