Are Graph Neural Networks Optimal Approximation Algorithms?

TL;DR

It is proved that polynomial-sized message-passing algorithms can represent the most powerful polynomial time algorithms for Max Constraint Satisfaction Problems assuming the Unique Games Conjecture.

Abstract

In this work we design graph neural network architectures that capture optimal approximation algorithms for a large class of combinatorial optimization problems, using powerful algorithmic tools from semidefinite programming (SDP). Concretely, we prove that polynomial-sized message-passing algorithms can represent the most powerful polynomial time algorithms for Max Constraint Satisfaction Problems assuming the Unique Games Conjecture. We leverage this result to construct efficient graph neural network architectures, OptGNN, that obtain high-quality approximate solutions on landmark combinatorial optimization problems such as Max-Cut, Min-Vertex-Cover, and Max-3-SAT. Our approach achieves strong empirical results across a wide range of real-world and synthetic datasets against solvers and neural baselines. Finally, we take advantage of OptGNN's ability to capture convex relaxations to design an algorithm for producing bounds on the optimal solution from the learned embeddings of OptGNN.

Figures (10)

• Figure 1: Schematic representation of OptGNN. During training, OptGNN produces node embeddings $\mathbf{v}$ using message passing updates on the graph $G$. These embeddings are used to compute the penalized objective $\mathcal{L}_p(\mathbf{v};G)$. OptGNN is trained by minimizing the average loss over the training set. At inference time, the fractional solutions (embeddings) $\mathbf{v}$ for an input graph $G$ produced by OptGNN are rounded using randomized rounding.
• Figure 2: Results for Max-Cut and Minimum Vertex Cover.
• Figure 3: Experimental comparison of SDP versus OptGNN Dual Certificates on random graphs of 100 nodes for the Max-Cut problem. Our OptGNN certificates track closely with the SDP certificates while taking considerably less time to generate.
• Figure 4: N=50 p=0.1 SDP vs OptGNN Dual Certificate
• Figure 5: Hyperparameter range explored for each group of datasets. For each NN architecture, when training on a dataset, we explored every listed hyperparameter combination in the corresponding column.

Theorems & Definitions (41)

• Definition : Quadratically Penalized Lagrangian
• Theorem 3.1
• Definition : OptGNN for Max-CSP
• Corollary 1: Informal
• Lemma 3.1: PAC learning
• Definition : OptGNN for 3-SAT
• Lemma C.1
• proof
• Theorem C.1
• proof