Annealed Mean Field Descent Is Highly Effective for Quadratic Unconstrained Binary Optimization

TL;DR

This work introduces Annealed Mean Field Descent (AMFD), a gradient-based method that directly minimizes the KL divergence between the mean-field approximation and the true QUBO distribution, addressing theoretical gaps in Mean Field Annealing. By tempering gradient updates with temperature and employing a second-order entropy approximation plus acceleration, AMFD achieves high-quality solutions across diverse COPs (MCP, MISP, TSP, QAP, GCP) with reduced problem dependence. Extensive experiments on five benchmark problems show AMFD often surpasses state-of-the-art QUBO solvers and competes favorably with Gurobi, highlighting its potential as a robust general-purpose solver for combinatorial optimization. The work argues that directly minimizing the KL divergence yields more accurate approximate distributions than the MFA self-consistency conditions, with practical impact in democratizing access to effective COP solving, potentially in hybrid with traditional optimization engines.

Abstract

In recent years, formulating various combinatorial optimization problems as Quadratic Unconstrained Binary Optimization (QUBO) has gained significant attention as a promising approach for efficiently obtaining optimal or near-optimal solutions. While QUBO offers a general-purpose framework, existing solvers often struggle with performance variability across different problems. This paper (i) theoretically analyzes Mean Field Annealing (MFA) and its variants--which are representative QUBO solvers, and reveals that their underlying self-consistent equations do not necessarily represent the minimum condition of the Kullback-Leibler divergence between the mean-field approximated distribution and the exact distribution, and (ii) proposes a novel method, the Annealed Mean Field Descent (AMFD), which is designed to address this limitation by directly minimizing the divergence. Through extensive experiments on five benchmark combinatorial optimization problems (Maximum Cut Problem, Maximum Independent Set Problem, Traveling Salesman Problem, Quadratic Assignment Problem, and Graph Coloring Problem), we demonstrate that AMFD exhibits superior performance in many cases and reduced problem dependence compared to state-of-the-art QUBO solvers and Gurobi--a state-of-the-art versatile mathematical optimization solver not limited to QUBO.

Figures (5)

• Figure 1: An approach to democratize combinatorial optimization by eliminating formulation and solver design.
• Figure 2: The difference between serial SA and parallel SA.
• Figure 3: Approximation of the mf-entropy and the derivative.
• Figure 4: Overview of this work.
• Figure 5: Comparison of QUBO solvers on a MCP, a MISP, a TSP, a QAP, and a GCP.