Rethinking and Accelerating Graph Condensation: A Training-Free Approach with Class Partition

TL;DR

This work tackles the high computational burden of training GNNs on large graphs by introducing CGC, a training-free graph condensation framework that shifts from class-to-class to class-to-node distribution matching. CGC leverages non-parametric feature propagation, data assessment, augmentation, and EM-based class partitioning to produce a compact, label-aware condensed graph with a closed-form feature generation step governed by a Dirichlet energy constraint, enabling CPU-only condensation. Empirical results show CGC and its graphless variant CGC-X deliver substantial speedups (up to 10^4×) over state-of-the-art GC methods while achieving higher or competitive accuracy across multiple datasets and GNN architectures, and with fewer hyper-parameters. The approach broadens GC applicability in practice by reducing computation, improving resilience to label sparsity, and enabling flexible integration with various propagation and partition strategies.

Abstract

The increasing prevalence of large-scale graphs poses a significant challenge for graph neural network training, attributed to their substantial computational requirements. In response, graph condensation (GC) emerges as a promising data-centric solution aiming to substitute the large graph with a small yet informative condensed graph to facilitate data-efficient GNN training. However, existing GC methods suffer from intricate optimization processes, necessitating excessive computing resources and training time. In this paper, we revisit existing GC optimization strategies and identify two pervasive issues therein: (1) various GC optimization strategies converge to coarse-grained class-level node feature matching between the original and condensed graphs; (2) existing GC methods rely on a Siamese graph network architecture that requires time-consuming bi-level optimization with iterative gradient computations. To overcome these issues, we propose a training-free GC framework termed Class-partitioned Graph Condensation (CGC), which refines the node distribution matching from the class-to-class paradigm into a novel class-to-node paradigm, transforming the GC optimization into a class partition problem which can be efficiently solved by any clustering methods. Moreover, CGC incorporates a pre-defined graph structure to enable a closed-form solution for condensed node features, eliminating the need for back-and-forth gradient descent in existing GC approaches. Extensive experiments demonstrate that CGC achieves an exceedingly efficient condensation process with advanced accuracy. Compared with the state-of-the-art GC methods, CGC condenses the Ogbn-products graph within 30 seconds, achieving a speedup ranging from $10^2$X to $10^4$X and increasing accuracy by up to 4.2%.

Figures (5)

• Figure 1: (a) The class-to-class matching paradigm in existing GC methods. (b) Our proposed class-to-node matching paradigm. $f$ denotes the relay model, $\mathcal{D}$ represents the distance function, and $g$ measures the matching objective (refer to Table \ref{['tab_GCcom']} for details).
• Figure 2: The pipeline of CGC and CGC-X.
• Figure 3: The accuracy and condensation time comparison of GC methods on Arxiv ($r=0.25\%$) and Reddit ($r=0.10\%$). SNTK and SNTK-X are out-of-memory on Reddit dataset.
• Figure 4: Test accuracy across varying hyper-parameters: the first row shows results for Cora ($r=2.60\%$), and the second row for Flickr ($r=0.50\%$).
• Figure 5: The performance on different feature propagation depths. GCN is trained on the condensed graph generated by CGC-X. $r$ is set as 0.25% and 0.10% for Arxiv and Reddit, respectively.

Theorems & Definitions (11)

• Proposition 1
• Proposition 2
• Proposition 3
• Remark 1
• Definition 1
• Remark 2
• Proposition 4
• Proposition 1
• Proposition 2
• Proposition 3