Enhancing GNNs Performance on Combinatorial Optimization by Recurrent Feature Update

TL;DR

This paper tackles NP-hard combinatorial optimization by formulating problems as QUBO and solving them with an unsupervised Graph Neural Network. It introduces QRF-GNN, a recurrent feature update framework that reuses previous iteration predictions as dynamic node features and uses parallel SAGE convolutions to iteratively minimize the QUBO relaxation $\mathcal{L}(\theta)=p(\theta)^T Q p(\theta)$. Across Max-Cut, Graph Coloring, and MIS, QRF-GNN outperforms leading learning-based baselines and matches or exceeds state-of-the-art heuristics on large instances, demonstrating strong scalability. The recurrent design reduces local minima entrapment and enables adaptive solving of CO problems, suggesting broad applicability to other QUBO-formulated tasks.

Abstract

Combinatorial optimization (CO) problems are crucial in various scientific and industrial applications. Recently, researchers have proposed using unsupervised Graph Neural Networks (GNNs) to address NP-hard combinatorial optimization problems, which can be reformulated as Quadratic Unconstrained Binary Optimization (QUBO) problems. GNNs have demonstrated high performance with nearly linear scalability and significantly outperformed classic heuristic-based algorithms in terms of computational efficiency on large-scale problems. However, when utilizing standard node features, GNNs tend to get trapped to suboptimal local minima of the energy landscape, resulting in low quality solutions. We introduce a novel algorithm, denoted hereafter as QRF-GNN, leveraging the power of GNNs to efficiently solve CO problems with QUBO formulation. It relies on unsupervised learning by minimizing the loss function derived from QUBO relaxation. The proposed key components of the architecture include the recurrent use of intermediate GNN predictions, parallel convolutional layers and combination of static node features as input. Altogether, it helps to adapt the intermediate solution candidate to minimize QUBO-based loss function, taking into account not only static graph features, but also intermediate predictions treated as dynamic, i.e. iteratively changing recurrent features. The performance of the proposed algorithm has been evaluated on the canonical benchmark datasets for maximum cut, graph coloring and maximum independent set problems. Results of experiments show that QRF-GNN drastically surpasses existing learning-based approaches and is comparable to the state-of-the-art conventional heuristics, improving their scalability on large instances.

Figures (10)

• Figure 1: The example of QRF-GNN work on a toy graph of 10 vertices and 12 edges on the Max-Cut problem. The Max-Cut involves partitioning of graph's nodes into two sets such that the number of edges between sets is maximized. At each iteration, the rounded (discretized) solution is shown: red color refers to $x_i = 0$, green to $x_i = 1$. Using the recurrent feature, QRF-GNN aims to reclassify each node to the opposite class of the majority of its nearest neighbors at each iteration, thereby optimizing the cut. On this instance, QRF-GNN is able to achieve the optimal solution in just a few iterations, while non-recurrent architectures (e.g., PI-GNN) require $\sim50$ times more iterations.
• Figure 2: The QRF-GNN architecture. Firstly, the problem graph is associated with the initial QUBO problem, and the node static features are extracted. Then the dynamic recurrent features from the previous iteration is concatenated with the static features. Finally, these fused input vectors along with the graph data pass through the graph neural network to update probabilities $p_i$ in Eq. \ref{['eq:loss-pi-gnn']}.
• Figure 3: Distribution of the results over 20 runs of GNNs with different convolutional layers with (left) and without (right) recurrent connection on several instances from Gset. Horizontal lines mean maximum, median and minimum values.
• Figure 4: a) The computation time of $5\times 10^4$ iterations of QRF-GNN on random regular graphs with $d=5$ in the sparse format depending on the number of nodes. b) The iteration number averaged over 20 graphs at which the algorithm found the best solution during the training process (green) and the mean P-value for 1 run (orange).
• Figure 5: Results distribution for 20 runs of the default QRF-GNN architecture depending on the learning rate for the Max-Cut problem on several instances from Gset. The number of iterations in all cases was fixed at $10^5$. Horizontal lines mean maximum, median and minimum values.