Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bonsai: Gradient-free Graph Condensation for Node Classification

Mridul Gupta, Samyak Jain, Vansh Ramani, Hariprasad Kodamana, Sayan Ranu

TL;DR

Bonsai tackles the scalability challenge of node classification with Graph Neural Networks by introducing a gradient-free graph condensation method. It represents the training space through computation trees and selects a small, representative exemplar set via reverse $k$-NN in Weisfeiler-Lehman space, followed by coverage-maximizing greedy selection and PPR-based sparsification to achieve a compact yet expressive condensed graph. The approach is model-agnostic and linear-time, yielding higher accuracy and substantially faster condensation than gradient-based baselines across seven real-world datasets, while also reducing energy consumption due to its CPU-bound design. This yields a practical, scalable alternative for training Gnns on large graphs and provides theoretical guarantees for the exemplar selection process. Bonsai’s release and reproducibility materials further strengthen its impact for developers and researchers seeking efficient graph data distillation.

Abstract

Graph condensation has emerged as a promising avenue to enable scalable training of GNNs by compressing the training dataset while preserving essential graph characteristics. Our study uncovers significant shortcomings in current graph condensation techniques. First, the majority of the algorithms paradoxically require training on the full dataset to perform condensation. Second, due to their gradient-emulating approach, these methods require fresh condensation for any change in hyperparameters or GNN architecture, limiting their flexibility and reusability. Finally, they fail to achieve substantial size reduction due to synthesizing fully-connected, edge-weighted graphs. To address these challenges, we present Bonsai, a novel graph condensation method empowered by the observation that \textit{computation trees} form the fundamental processing units of message-passing GNNs. Bonsai condenses datasets by encoding a careful selection of \textit{exemplar} trees that maximize the representation of all computation trees in the training set. This unique approach imparts Bonsai as the first linear-time, model-agnostic graph condensation algorithm for node classification that outperforms existing baselines across $7$ real-world datasets on accuracy, while being $22$ times faster on average. Bonsai is grounded in rigorous mathematical guarantees on the adopted approximation strategies making it robust to GNN architectures, datasets, and parameters.

Bonsai: Gradient-free Graph Condensation for Node Classification

TL;DR

Bonsai tackles the scalability challenge of node classification with Graph Neural Networks by introducing a gradient-free graph condensation method. It represents the training space through computation trees and selects a small, representative exemplar set via reverse -NN in Weisfeiler-Lehman space, followed by coverage-maximizing greedy selection and PPR-based sparsification to achieve a compact yet expressive condensed graph. The approach is model-agnostic and linear-time, yielding higher accuracy and substantially faster condensation than gradient-based baselines across seven real-world datasets, while also reducing energy consumption due to its CPU-bound design. This yields a practical, scalable alternative for training Gnns on large graphs and provides theoretical guarantees for the exemplar selection process. Bonsai’s release and reproducibility materials further strengthen its impact for developers and researchers seeking efficient graph data distillation.

Abstract

Graph condensation has emerged as a promising avenue to enable scalable training of GNNs by compressing the training dataset while preserving essential graph characteristics. Our study uncovers significant shortcomings in current graph condensation techniques. First, the majority of the algorithms paradoxically require training on the full dataset to perform condensation. Second, due to their gradient-emulating approach, these methods require fresh condensation for any change in hyperparameters or GNN architecture, limiting their flexibility and reusability. Finally, they fail to achieve substantial size reduction due to synthesizing fully-connected, edge-weighted graphs. To address these challenges, we present Bonsai, a novel graph condensation method empowered by the observation that \textit{computation trees} form the fundamental processing units of message-passing GNNs. Bonsai condenses datasets by encoding a careful selection of \textit{exemplar} trees that maximize the representation of all computation trees in the training set. This unique approach imparts Bonsai as the first linear-time, model-agnostic graph condensation algorithm for node classification that outperforms existing baselines across real-world datasets on accuracy, while being times faster on average. Bonsai is grounded in rigorous mathematical guarantees on the adopted approximation strategies making it robust to GNN architectures, datasets, and parameters.

Paper Structure

This paper contains 32 sections, 6 theorems, 18 equations, 10 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction and Related Works
  2. Existing works and their Limitations
  3. Contributions
  4. Problem Formulation and Preliminaries
  5. Computation structure of Gnns
  6. Bonsai: Proposed Methodology
  7. Quantifying Representativeness through Reverse $k$-NN
  8. Sampling for Scalable Computation of Reverse $k$-NN
  9. Coverage Maximization
  10. Properties of Bonsai
  11. Experiments
  12. Experimental Setup
  13. Prediction Accuracy
  14. Cross-architecture Generalization
  15. Condensation Efficiency
  16. ...and 17 more sections

Key Result

Corollary 1

If $\mathcal{T}\xspace_v^L$ is isomorphic to $\mathcal{T}\xspace_u^L$, then $\mathbf{a}\xspace_v^L=\mathbf{a}\xspace_u^L$.

Figures (10)

  • Figure 1: The figure depicts the construction of computation trees for nodes $v_1$ and $v_ {11}$ in the sample graph, at depth $L=2$. Node colors indicate their labels. Despite being distant from each other in the graph and embedded in non-isomorphic $L$-hop neighborhoods, $v_1$ and $v_{11}$ have isomorphic computation trees.
  • Figure 2: Pearson correlations between the $L_2$ distances of node pairs in the Gnn embedding space and the unsupervised embeddings derived from the WL-kernel, computed for pairs with a specific distance threshold in the WL-space ($x$-axis).
  • Figure 3: Pipeline of the proposed algorithm for graph condensation using a toy example is displayed. We use $L=1$ and $k=2$ for this example.
  • Figure 4: Ablation study of Bonsai.
  • Figure 5: Distribution of PPR scores against node ranks.
  • ...and 5 more figures

Theorems & Definitions (14)

  • Definition 1: Graph
  • Definition 2: Graph Isomorphism
  • Definition 3: Computation Tree
  • Definition 4: Weisfeiler-Lehman (WL) Kernel wlkernel
  • Corollary 1
  • Definition 5: Reverse $k$-NN and Representative Power
  • Lemma 1
  • Definition 6: The exemplars
  • Theorem 1
  • Theorem 2
  • ...and 4 more