Binarizing Physics-Inspired GNNs for Combinatorial Optimization

TL;DR

This work analyzes physics-inspired GNNs for combinatorial optimization and identifies a density-driven phase transition that pushes the relaxed, continuous solutions away from meaningful binary decisions. It introduces a fuzzy-QUBO interpretation and binarization strategies, including temperature annealing and straight-through gradient estimation, to stabilize training and improve performance on dense graphs. Empirical results on MaxCut and MIS show significant gains over the baseline PI-GNNs, especially as graph density increases, validating the viability of principled discretization for CO with GNNs. The findings highlight a practical pathway to scale PI-GNNs to denser, more realistic CO problems while maintaining GPU efficiency and interpretability through discrete activations.

Abstract

Physics-inspired graph neural networks (PI-GNNs) have been utilized as an efficient unsupervised framework for relaxing combinatorial optimization problems encoded through a specific graph structure and loss, reflecting dependencies between the problem's variables. While the framework has yielded promising results in various combinatorial problems, we show that the performance of PI-GNNs systematically plummets with an increasing density of the combinatorial problem graphs. Our analysis reveals an interesting phase transition in the PI-GNNs' training dynamics, associated with degenerate solutions for the denser problems, highlighting a discrepancy between the relaxed, real-valued model outputs and the binary-valued problem solutions. To address the discrepancy, we propose principled alternatives to the naive strategy used in PI-GNNs by building on insights from fuzzy logic and binarized neural networks. Our experiments demonstrate that the portfolio of proposed methods significantly improves the performance of PI-GNNs in increasingly dense settings.

Figures (8)

• Figure 1: Schematic of an Ising spin chain in 1D, with arrow/spin down corresponding to assignment $0$, and up to $1$, respectively.
• Figure 2: Schematic of the QUBO and Ising spin models.
• Figure 3: Distribution of post-activation values ($a^{post}$) over training epochs on the MaxCut problem with sparser graphs.
• Figure 4: Distribution of (a) pre-activations ($a^{pre}$) and (b) post-activations ($a^{post}$) over training epochs on the MaxCut problem with denser graphs.
• Figure 5: Schema of the PI-GNNs pipeline for solving CO under QUBO formalization (problem encoding, graph convolutions, node assignments), showcased on a MaxCut problem. The proposed modifications---binarization of the ultimate nonlinearity (\ref{['sec:discretization']}) and loss fuzzification (\ref{['sec:fuzzy']})---are highlighted in blue.

Theorems & Definitions (4)

• Definition 1: Combinatorial Optimization Problem
• Definition 2: Maximum Cut
• Definition 3: Maximum Independent Set
• Definition 4: Fuzzy Conjunction