Perpertual Coupled Simulated Annealing for Continuous Optimization

TL;DR

The paper addresses the sensitivity of dispersion control in global optimization by coupling multiple Simulated Annealing processes (CSA) and introducing Perpetual Orbit (PO) to autonomously manage the dispersion variable. PO-CSA replaces manual scheduling of the generation temperature with a dynamic, parameter-free mechanism that keeps exploration active around the best dispersion value, while preserving CSA’s variance-based acceptance control. Across 14 benchmark functions, PO-CSA yields equal or better solutions than classical heuristics and outperforms CSA variants that rely on predefined initial temperatures, demonstrating strong robustness to initialization and budgets. The approach promises practical impact by reducing metaheuristic tuning requirements and offering a versatile mechanism applicable to other ensemble methods.

Abstract

Global optimization heuristics are popular to optimize hard non-convex problems. Despite their irrefutably large cost-to-solution, in the lack of other working greedy or convex approaches, global optimization algorithms remain the no-brainer choice. Nevertheless, successful use often requires tedious adjustments to initial parameters to avoid premature convergence. The Coupled Simulated Annealing approach proposed a method based on the coupling of multiple optimizers to escape premature convergence, having achieved success in optimizing hyperparameters of many applications of machine learning; however, a careful choice of the generation temperature is still required. In this paper we propose the Perpetual Orbit technique as a solution to control the generation temperature and avoid search stagnation. In principle, this technique can also be applied to other ensemble- and population-based algorithms that have a dispersion variable. The results of our experiments show superior performance when using the proposed technique because it makes the Couple Simulated Annealing totally parameter-free and capable of reaching equal or better solutions in more than 85\% cases across all functions and competitor methods analyzed.

Figures (4)

• Figure 1: The Perpetual Orbit technique. (a) The standard behavior; and (b) The increase of $U^{s_2}_{k+1}$ after $V^{s_2}_{k+1}$ reaches the upper bound.
• Figure 2: The PO-CSA behavior on $f_3$ along 10 thousand iterations with $D=10$ for three different initial generation temperatures. The number of optimizers equals $D$. After a transient, all three runs converge to the same level of energy. (a) The reference for the generation temperature, (b) The generation temperature for all optimizers.
• Figure 3: The best PO-CSA energies on $f_3$ along 10 thousand iterations with $D=10$ for three different initial generation temperatures. The number of optimizers equals $D$. After a short transient, all three runs converge to the same level of energy.
• Figure 4: Average energy of $f_2$ over 25 runs for different numbers of function evaluations per optimizer. The number of optimizers is the same as the dimension. In (d) and (e) for $D=5$, and (f) for $D=5,10$, there are no points for PO-CSA because its costs are zero (global optimum) and not defined in logarithmic scale. The larger the number of function evaluations, the better the results of the PO-CSA because it does not stagnates its search.