Graph-SCP: Accelerating Set Cover Problems with Graph Neural Networks

TL;DR

This work addresses speeding up the Set Cover Problem (SCP) by learning to identify a small subproblem that contains the solution space. It introduces Graph-SCP, a graph neural network framework that augments SCP graphs with rich feature sets and learns to predict a subgraph to hand to a traditional solver, achieving up to $10$x speedups while preserving optimality. The method combines supervised labels from solved instances with an unsupervised LP-relaxation objective and uses iterative thresholding with warm-starts to refine the subproblem, demonstrating strong generalization to larger instances and to OR Library benchmarks. Practically, Graph-SCP can be plugged into existing solvers (e.g., Gurobi or SCIP), significantly accelerating SCP solving across varied densities and problem sizes without sacrificing solution quality.

Abstract

Machine learning (ML) approaches are increasingly being used to accelerate combinatorial optimization (CO) problems. We investigate the Set Cover Problem (SCP) and propose Graph-SCP, a graph neural network method that augments existing optimization solvers by learning to identify a smaller sub-problem that contains the solution space. Graph-SCP uses both supervised learning from prior solved instances and unsupervised learning to minimize the SCP objective. We evaluate the performance of Graph-SCP on synthetically weighted and unweighted SCP instances with diverse problem characteristics and complexities, and on instances from the OR Library, a canonical benchmark for SCP. We show that Graph-SCP reduces the problem size by 60-80% and achieves runtime speedups of up to 10x on average when compared to Gurobi (a state-of-the-art commercial solver), while maintaining solution quality. This is in contrast to fast greedy solutions that significantly compromise solution quality to achieve guaranteed polynomial runtime. We showcase Graph-SCP's ability to generalize to larger problem sizes, training on SCP instances with up to 3,000 subsets and testing on SCP instances with up to 10,000 subsets.

Figures (8)

• Figure 1: (A) A simple example of a set cover problem with equal cost subsets. (B) The set cover instance represented using a bipartite graph abstraction. This graph is passed into a GNN which then predicts whether nodes (i.e. subsets) belong to the SCP solution (green highlighted nodes). The prediction values for Layer 0 and Layer 1 nodes (white-colored nodes) are discarded, since only Layer 2 nodes contribute to the solution. (C) The hypergraph representation for the same set cover instance. Each subset corresponds to a hyperedge shown as bounding boxes around the element nodes. (D) For each hyperedge $e$, we define hyperedge dependent weights $\gamma_{e}(v)$ for each element $v$ of the universe as the inverse of the size of its associated hyperedge as shown.
• Figure 2: Graph-SCP takes an SCP instance represented as a graph as input. The graph is augmented with a set of features on its nodes ($G_f$). A GNN is trained on this graph to predict a subgraph of nodes that likely contains the solution to the SCP instance. At runtime, the GNN is used (only once) to generate predictions. Graph-SCP picks nodes at a predefined percentile threshold as the subgraph. If the objective criteria are not met, the threshold is reduced, thus selecting a larger subgraph.
• Figure 3: Visualization of node embeddings in two dimensions using PaCMAP pacmap. The embeddings are extracted from the penultimate layer of Graph-SCP's message passing GNN with different sets of node features. The features used are displayed above each column. Each row shows embeddings for a different type of SCP instance (see Table \ref{['instance_table']}). To quantify how well various sets of features can separate solution nodes from non-solution nodes, we measure the average distance of a solution node to other solution nodes (intra distance) and the average distance of a solution node to non-solution nodes (inter distance) in the GNN embedding space. Observe how (A) and (E) have the largest inter distance with small intra distances, which indicates that Cost, Cover, RWR, degree-based, and hypergraph-based features are useful in separating solution nodes from non-solution nodes.
• Figure 4: Graph-SCP achieves the optimal objective with faster runtimes. We see the largest speedup of about 70x for Instance Type 2 (B) and the smallest speedup of about 3x for Instance Type 4 (D). Note that the greedy algorithm runs significantly faster across all 4 instances, but obtains poor objective values. In each subplot, the area above the red line (random sample) correspond to the benefits of learning. Thirty instances were used across each type with error bars showing standard error.
• Figure 5: Generalization properties of Graph-SCP against problem instances with complexities as defined in gasse2019exact. Here, $m$ is the size of the universe and $n$ is the number of subsets, with densities $d=0.05$. We use the same model trained on instances of Types 1 through 4, that are smaller with a maximum of 400 rows. Graph-SCP achieves faster speedups in runtimes when compared to GCNN Branching. Ten instances were sampled for each problem type with error bars showing standard error.