Reheated Gradient-based Discrete Sampling for Combinatorial Optimization

TL;DR

This paper identifies wandering in contours as a fundamental inefficiency in gradient-based discrete samplers for combinatorial optimization. It introduces ReSCO, a reheating mechanism that resets the SA temperature to a theoretically informed critical temperature based on specific heat, enabling better exploration after wandering begins. The authors provide extensive experiments across MIS, MC, MaxCut, and Graph Balanced Partition showing that ReSCO improves solution quality with minimal overhead and broad applicability to gradient-based samplers. The work offers a practical, theoretically motivated approach to balance exploration and exploitation in discrete gradient-based CO solvers, with strong empirical gains and directions for future theoretical analysis.

Abstract

Recently, gradient-based discrete sampling has emerged as a highly efficient, general-purpose solver for various combinatorial optimization (CO) problems, achieving performance comparable to or surpassing the popular data-driven approaches. However, we identify a critical issue in these methods, which we term ''wandering in contours''. This behavior refers to sampling new different solutions that share very similar objective values for a long time, leading to computational inefficiency and suboptimal exploration of potential solutions. In this paper, we introduce a novel reheating mechanism inspired by the concept of critical temperature and specific heat in physics, aimed at overcoming this limitation. Empirically, our method demonstrates superiority over existing sampling-based and data-driven algorithms across a diverse array of CO problems.

Figures (8)

• Figure 1: Visualization of the "wandering in contours" phenomenon.
• Figure 1: Left: Average approximation ratio of different methods on two MaxClique tasks, where iSCO-N and ReSCO-N represent running N sampling chains for each problem; Right: Runtime of single-chain iSCO, single-chain ReSCO, and 32-chain iSCO on MIS tasks on a machine with a single NVIDIA RTX A6000 GPU.
• Figure 2: The wandering in contours phenomenon of a gradient-based discrete sampler on a maximum independent set problem. (a): The trajectory of objective values of traditional simulated annealing, where the algorithm updates the value slowly. (b): The trajectory of objective values of gradient-based sampling methods. The algorithm converges rapidly to a stop value solution (highlighted in green) and then stays close to this value. (c): The objective values after the stop point continue to fluctuate but stay very similar to or the same as the stop value. (d): Gradient-based samplers produce solutions with similar objective values but large distances from the stop point. (e): All the newly sampled solutions in traditional simulated annealing are near each other.
• Figure 3: (a): Illustration of the discrete optimization landscape for the function $\frac{1}{4} x^4 - \frac{4}{3} x^3 + \frac{15}{8} x^2$. Directed by the gradient information, the gradient-based discrete samplers will oscillate between two discrete solutions $2$ and $3$ near the local minimum $2.5$. (b): Visualization of $f(x_1,x_2)$ from \ref{['eq:toy_case']}. In the experiment, we start from the local minimum $(0,0)$ (marked as green) and test whether the discrete samplers can reach the global minimum $(1,1)$ (marked as red); (c): The average escaping rate of three gradient-based discrete samplers, which is defined as the probability of escaping from local minimum $(0,0)$ to the global minimum $(1,1)$ by running 20 sampling steps at different temperatures, declines sharply as temperature decreases.
• Figure 4: (a): Comparison of the first 200 steps shows that SA with PAS, a gradient-based discrete sampler, achieves fast initial improvement, then slows, unlike traditional SA's steady progress. (b): From steps 100 to 600, SA with PAS initially has much higher specific heat, which then quickly drops and stabilizes, in contrast to SA's steady specific heat levels. (c): After converging to the stop value point, iSCO begins to wander in contours and wastes the rest of the steps. (d): Upon detecting "wandering in contours", ReSCO reheats to the critical temperature, facilitating an escape from this behavior and discovering better solutions.