Subsampling Graphs with GNN Performance Guarantees
Mika Sarkin Jain, Stefanie Jegelka, Ishani Karmarkar, Luana Ruiz, Ellen Vitercik
TL;DR
The paper addresses the challenge of training GNNs on large graph datasets by proposing principled subsampling with provable guarantees. It introduces a coreset construction approach based on Tree Mover's Distance (TMD) that subsamples both whole graphs and nodes in a model- and label-agnostic manner, alongside a linear-time method to compute a graph’s tree-norm. The authors prove bounds on the loss increase when training on the subsample and demonstrate strong empirical performance against baselines across diverse datasets, highlighting reduced labeling and computational costs without sacrificing accuracy. This work enables early-stage data reduction in graph learning pipelines and paves the way for robust, scalable GNN training. The results have practical implications for domains with expensive labeling and large graph collections, offering a theoretically grounded and empirically effective subsampling framework.
Abstract
How can we subsample graph data so that a graph neural network (GNN) trained on the subsample achieves performance comparable to training on the full dataset? This question is of fundamental interest, as smaller datasets reduce labeling costs, storage requirements, and computational resources needed for training. Selecting an effective subset is challenging: a poorly chosen subsample can severely degrade model performance, and empirically testing multiple subsets for quality obviates the benefits of subsampling. Therefore, it is critical that subsampling comes with guarantees on model performance. In this work, we introduce new subsampling methods for graph datasets that leverage the Tree Mover's Distance to reduce both the number of graphs and the size of individual graphs. To our knowledge, our approach is the first that is supported by rigorous theoretical guarantees: we prove that training a GNN on the subsampled data results in a bounded increase in loss compared to training on the full dataset. Unlike existing methods, our approach is both model-agnostic, requiring minimal assumptions about the GNN architecture, and label-agnostic, eliminating the need to label the full training set. This enables subsampling early in the model development pipeline (before data annotation, model selection, and hyperparameter tuning) reducing costs and resources needed for storage, labeling, and training. We validate our theoretical results with experiments showing that our approach outperforms existing subsampling methods across multiple datasets.