Kinetic simulated annealing optimization with entropy-based cooling rate

TL;DR

This work develops a kinetic, gradient-free simulated annealing framework with a dynamic, state-dependent cooling temperature. By formulating the particle dynamics as an interacting-particle system and deriving a Boltzmann-type equation in an extended phase space, the authors establish a mean-field (Fokker–Planck) limit under quasi-invariant scaling and prove exponential entropy decay via a carefully designed closed-loop feedback $\lambda[f]$. The temperature evolves according to a coupled Fokker–Planck equation, yielding quasi-equilibria that depend on the temperature distribution and the current state. Numerical validation using DSMC/EntKSA demonstrates accelerated convergence to global minima for both $k=1$ and $k>1$ cases, with the temperature dynamics adopting generalized gamma profiles and enabling faster entropy dissipation compared with classical SA.

Abstract

We present a modified simulated annealing method with a dynamical choice of the cooling temperature. The latter is determined via a closed-loop control and is proven to yield exponential decay of the entropy of the particle system. The analysis is carried out through kinetic equations for interacting particle systems describing the simulated annealing method in an extended phase space. Decay estimates are derived under the quasi-invariant scaling of the resulting system of Boltzmann-type equations to assess the consistency with their mean-field limit. Numerical results are provided to illustrate and support the theoretical findings.

Figures (10)

• Figure 1: Top row: evolution of the Shannon entropy $H(f|f^q)(t)$ for several values of $\alpha$ and for $\epsilon = 10^{-2}$ (left) and $\epsilon = 10^{-3}$ (right), we have reported in green approximated evolution of the KSA algorithm with $\epsilon = 10^{-4}$. Bottom row: evolution of the mean temperature of the system of particles $m_1(t) = \int_{\mathbb R_+}Tg(T,t)dT$ for $\epsilon = 10^{-2}$ (left) and $\epsilon = 10^{-3}$ (right).
• Figure 2: Evolution of the distribution $f(x,t)$ at time $t = 1$ for several values of $\alpha = 0.025,0.05,0.1$ and for $\epsilon = 10^{-2}$ (left) or $\epsilon= 10^{-3}$ (right). In dotted green, we highlighted $x^*$ corresponding to the real minimum of the function $\mathcal{F}(x)$.
• Figure 3: Evolution of the distributions $f(x,t)$, $g(T,t)$ at time $t = 1,5,10$ for several values of $\alpha = 0.025$ (left) $\alpha = 0.05$ (center), and $\alpha = 0.1$ (right) for a fixed $\epsilon= 10^{-3}$. In the top row, we report the evolution of $f(x,t)$, and in the bottom row the evolution of $g(T,t)$. We considered $N = 10^6$ particles both in space and temperature, $p= 1/4$, $\theta = 0.05$ and $\sigma^2 = 0.1$.
• Figure 4: Evolution of the mean position $m_x = \int_{\mathbb R} x f(x,t)dx$ and of its variance $Var_x = \int_{\mathbb R} (x-m_x)^2 f(x,t)dx$ over the time interval $[0,50]$ and several values of the parameter $\alpha = 0.025,0.05,0.1$.
• Figure 5: Top row: estimated values of $\lambda>0$ defined in \ref{['eq:lambda_kappa1']} and several values of $\alpha = 0.025,0.05,0.1$. Bottom row: estimated values of $\mathcal{I}_\mathcal{F}(t) = \int_{\mathbb R} \mathcal{F}(x)(f^q(x,t)-f(x,t))dx$ for several values of $\alpha = 0.025,0.05,0.1$. We considered $\epsilon = 10^{-2}$ (left) and $\epsilon = 10^{-3}$ (right).

Theorems & Definitions (7)

• Remark 2.1
• Remark 2.2
• Remark 2.3
• Lemma 3.1: Case $k=1$
• Remark 3.2
• Lemma 3.3: Case $k>1$
• Remark 3.4