Regularized Langevin Dynamics for Combinatorial Optimization
Shengyu Feng, Yiming Yang
TL;DR
The paper advances combinatorial optimization by introducing Regularized Langevin Dynamics (RLD), which enforces a fixed expected update distance to mitigate local optima in discrete spaces. It yields two solvers: Regularized Langevin Simulated Annealing (RLSA) and Regularized Langevin Neural Network (RLNN), with RLSA delivering substantial runtime reductions and competitive results, and RLNN offering efficient, unsupervised training. Across MIS, MCl, and MCut benchmarks, RLD-based methods achieve state-of-the-art or near-state-of-the-art performance while reducing computational costs. The approach is simple, generalizable to multiple CO problems, and highlights a promising direction for combining gradient-informed sampling with distance-based regularization in discrete optimization.
Abstract
This work proposes a simple yet effective sampling framework for combinatorial optimization (CO). Our method builds on discrete Langevin dynamics (LD), an efficient gradient-guided generative paradigm. However, we observe that directly applying LD often leads to limited exploration. To overcome this limitation, we propose the Regularized Langevin Dynamics (RLD), which enforces an expected distance between the sampled and current solutions, effectively avoiding local minima. We develop two CO solvers on top of RLD, one based on simulated annealing (SA), and the other one based on neural network (NN). Empirical results on three classic CO problems demonstrate that both of our methods can achieve comparable or better performance against the previous state-of-the-art (SOTA) SA- and NN-based solvers. In particular, our SA algorithm reduces the runtime of the previous SOTA SA method by up to 80\%, while achieving equal or superior performance. In summary, RLD offers a promising framework for enhancing both traditional heuristics and NN models to solve CO problems. Our code is available at https://github.com/Shengyu-Feng/RLD4CO.