Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Evolutionary Optimization Using the Ising Model

Simon Klüttermann

TL;DR

This paper tackles the challenge of finding global minima on highly multimodal, potentially non-differentiable loss surfaces. It introduces an Ising-model-inspired evolutionary optimization that forms semi-stable regions of solutions and uses temperature-controlled updates to explore across regions, aiming for robust global minimization. Empirical results on a designed test function show the Ising-based approach achieving near-optimal minima (around $$2.2\times 10^{-5}$$) and delivering diverse, region-based subsolutions that can benefit ensembles. The findings suggest a practical impact in data mining by enabling efficient global search and directly furnishing ensemble-relevant diversity without multiple full optimizations, with future work exploring higher-dimensional regions and broader loss-function applicability.

Abstract

In this paper, we study the problem of finding the global minima of a given function. Specifically, we consider complicated functions with numerous local minima, as is often the case for real-world data mining losses. We do so by applying a model from theoretical physics to create an Ising model-based evolutionary optimization algorithm. Our algorithm creates stable regions of local optima and a high potential for improvement between these regions. This enables the accurate identification of global minima, surpassing comparable methods, and has promising applications to ensembles.

Towards Evolutionary Optimization Using the Ising Model

TL;DR

This paper tackles the challenge of finding global minima on highly multimodal, potentially non-differentiable loss surfaces. It introduces an Ising-model-inspired evolutionary optimization that forms semi-stable regions of solutions and uses temperature-controlled updates to explore across regions, aiming for robust global minimization. Empirical results on a designed test function show the Ising-based approach achieving near-optimal minima (around ) and delivering diverse, region-based subsolutions that can benefit ensembles. The findings suggest a practical impact in data mining by enabling efficient global search and directly furnishing ensemble-relevant diversity without multiple full optimizations, with future work exploring higher-dimensional regions and broader loss-function applicability.

Abstract

In this paper, we study the problem of finding the global minima of a given function. Specifically, we consider complicated functions with numerous local minima, as is often the case for real-world data mining losses. We do so by applying a model from theoretical physics to create an Ising model-based evolutionary optimization algorithm. Our algorithm creates stable regions of local optima and a high potential for improvement between these regions. This enables the accurate identification of global minima, surpassing comparable methods, and has promising applications to ensembles.

Paper Structure

This paper contains 13 sections, 1 theorem, 5 equations, 8 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Evolutionary Optimization
  4. Ising Model
  5. Science-Inspired Machine learning
  6. Ensemble methods
  7. Ising based Evolution
  8. Comparison algorithms
  9. Test Description
  10. Results
  11. Phase transitions
  12. Ensembles
  13. Conclusion

Key Result

Theorem 1

The only global optimum to function eqn:thefunc is $x=-1$.

Figures (8)

  • Figure 1: Visualization of Eq. \ref{['eqn:thefunc']}. Because of the double logarithmic axes, the sinus looks unusual. But this highlights that the local minima are not randomly distributed.
  • Figure 2: Distribution of locally minimal function values.
  • Figure 3: Average minimal value found for different evolutionary optimization algorithms.
  • Figure 4: Average lowest value found as a function of the number of function evaluations used. We zoom in on this region because the most interesting changes happen in the lower right corner. We also add the lowest three possible solutions of Eq. \ref{['eqn:thefunc']} to be found as horizontal lines.
  • Figure 5: Current state (color) over the population, as a temperature and update step function.
  • ...and 3 more figures

Theorems & Definitions (2)

  • Theorem 1
  • proof