Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Combinatorial Optimization via Heat Diffusion

Hengyuan Ma, Wenlian Lu, Jianfeng Feng

TL;DR

This study proposes a framework for solving general combinatorial optimization problems by enabling information to actively propagate to the solver through heat diffusion, and reveals its significant potential in advancing combinatorial optimization.

Abstract

Combinatorial optimization problems are widespread but inherently challenging due to their discrete nature. The primary limitation of existing methods is that they can only access a small fraction of the solution space at each iteration, resulting in limited efficiency for searching the global optimal. To overcome this challenge, diverging from conventional efforts of expanding the solver's search scope, we focus on enabling information to actively propagate to the solver through heat diffusion. By transforming the target function while preserving its optima, heat diffusion facilitates information flow from distant regions to the solver, providing more efficient navigation. Utilizing heat diffusion, we propose a framework for solving general combinatorial optimization problems. The proposed methodology demonstrates superior performance across a range of the most challenging and widely encountered combinatorial optimizations. Echoing recent advancements in harnessing thermodynamics for generative artificial intelligence, our study further reveals its significant potential in advancing combinatorial optimization.

Efficient Combinatorial Optimization via Heat Diffusion

TL;DR

This study proposes a framework for solving general combinatorial optimization problems by enabling information to actively propagate to the solver through heat diffusion, and reveals its significant potential in advancing combinatorial optimization.

Abstract

Combinatorial optimization problems are widespread but inherently challenging due to their discrete nature. The primary limitation of existing methods is that they can only access a small fraction of the solution space at each iteration, resulting in limited efficiency for searching the global optimal. To overcome this challenge, diverging from conventional efforts of expanding the solver's search scope, we focus on enabling information to actively propagate to the solver through heat diffusion. By transforming the target function while preserving its optima, heat diffusion facilitates information flow from distant regions to the solver, providing more efficient navigation. Utilizing heat diffusion, we propose a framework for solving general combinatorial optimization problems. The proposed methodology demonstrates superior performance across a range of the most challenging and widely encountered combinatorial optimizations. Echoing recent advancements in harnessing thermodynamics for generative artificial intelligence, our study further reveals its significant potential in advancing combinatorial optimization.
Paper Structure (20 sections, 4 theorems, 59 equations, 10 figures, 2 tables, 6 algorithms)

This paper contains 20 sections, 4 theorems, 59 equations, 10 figures, 2 tables, 6 algorithms.

Table of Contents

  1. Introduction
  2. Failure of gradient-based combinatorial optimization
  3. Monte Carlo gradient estimation
  4. Heat diffusion optimization
  5. Heat diffusion on the parameter space
  6. Solving the heat equation
  7. Proposed algorithm
  8. Experiments
  9. Discussion
  10. Conclusion
  11. Proof of theorems
  12. Proof of Thm. \ref{['thm:consistent']}
  13. Proof of Thm. \ref{['thm:erf']}
  14. Proof of Thm. \ref{['thm:control']}
  15. Complexity analysis
  16. ...and 5 more sections

Key Result

Theorem 1

For any $\tau > 0$, the function $u(\tau,\bm{\theta})$ and $h(\bm{\theta})$ has the same global minima in $\bar{\mathbb{R}}^n$

Figures (10)

  • Figure 1: The Heat diffusion optimization (HeO) framework. The efficiency of searching a key in a dark room is significantly improved by employing navigation that utilizes heat emission from the key. In combinatorial optimization, heat diffusion transforms the target function into different versions while preserving the location of the optima. Therefore, the gradient information of these transformed functions cooperatively help to optimize the original target function.
  • Figure 2: Performance of HeO (Alg. \ref{['alg:heo_mon']}, Appendix), Monte Carlo gradient estimation (MCGE), Hopfield neural network (HNN) and simulated annealing (SA) on minimizing the output of a neural network (Eq. \ref{['eq:ran_nn']}). Top panel: the energy ($f(\cdot)$). Bottom panel: the uncertainty $V(\bm{\theta})$ (Eq. \ref{['eq:variance']}).
  • Figure 3: a, Illustration of the max-cut problem. b, Performance of HeO (Alg. \ref{['alg:heo']}) and representative iterative approximation methods including LQA bowles2022quadratic, aSB goto2019combinatorial, bSB goto2021high, dSB goto2021high, CIM wang2013coherent and SIM-CIM tiunov2019annealing on max-cut problems from the Biq Mac Library wiegele2007biq. Top panel: average relative loss for each algorithm over all problems. Bottom panel: the count of instances where each algorithm ended up with one of the bottom-2 worst results among the 7 algorithms.
  • Figure 4: a, Illustration of the Boolean 3-satisfiability (3-SAT) problem. b, Performance of HeO (Alg. \ref{['alg:sat']}, Appendix), 2-order and 3-order oscillation Ising machine (OIM) bybee2023efficient on 3-SAT problems with various number of variables from the SATLIB hoos2000satlib. We report the mean percent of constraints satisfied (left) and probability of satisfying all claims (right) for each algorithm.
  • Figure 5: a, Training a network with ternary-value parameters. b, The weight value accuracy of the HeO (Alg. \ref{['alg:nn']}, Appendix) and Monte Carlo gradient estimation (MCGE) with momentum under different sizes of training set. ($n=100,m=1,5,20$). For each test, we estimate the mean from 10 runs.
  • ...and 5 more figures

Theorems & Definitions (6)

  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Theorem S4
  • proof
  • proof