Disentangled Condensation for Large-scale Graphs

TL;DR

DisCo tackles the scalability bottleneck of graph condensation by decoupling the process into node condensation and edge translation, entirely avoiding joint optimization of nodes, edges, and GNNs. It uses a pre-trained MLP with class centroid alignment and anchor attachments for node condensation, and a neighbor-aware link predictor transferred via anchors for edge generation. The two-stage, GNN-free design yields substantial speedups and scales to graphs with over $10^8$ nodes, while maintaining or improving downstream accuracy on five common datasets and the Ogbn-papers100M benchmark. Empirically, DisCo demonstrates strong generalizability across GNN architectures and achieves higher fidelity at larger scale than state-of-the-art methods, making graph condensation practical for industrial-scale graphs.

Abstract

Graph condensation has emerged as an intriguing technique to save the expensive training costs of Graph Neural Networks (GNNs) by substituting a condensed small graph with the original graph. Despite the promising results achieved, previous methods usually employ an entangled paradigm of redundant parameters (nodes, edges, GNNs), which incurs complex joint optimization during condensation. This paradigm has considerably impeded the scalability of graph condensation, making it challenging to condense extremely large-scale graphs and generate high-fidelity condensed graphs. Therefore, we propose to disentangle the condensation process into a two-stage GNN-free paradigm, independently condensing nodes and generating edges while eliminating the need to optimize GNNs at the same time. The node condensation module avoids the complexity of GNNs by focusing on node feature alignment with anchors of the original graph, while the edge translation module constructs the edges of the condensed nodes by transferring the original structure knowledge with neighborhood anchors. This simple yet effective approach achieves at least 10 times faster than state-of-the-art methods with comparable accuracy on medium-scale graphs. Moreover, the proposed DisCo can successfully scale up to the Ogbn-papers100M graph containing over 100 million nodes with flexible reduction rates and improves performance on the second-largest Ogbn-products dataset by over 5%. Extensive downstream tasks and ablation study on five common datasets further demonstrate the effectiveness of the proposed DisCo framework. Our code is available at https://github.com/BangHonor/DisCo.

Figures (6)

• Figure 1: The GPU requirements and accuracy of different condensation methods on the second largest Ogbn-products dataset, where "0.02%", "0.04%", "0.08%" refer to the reduction rates.
• Figure 2: An illustrative diagram of the proposed DisCo framework.
• Figure 3: The performance comparison of DisCo and baselines using GCN under different reduction rates on Ogbn-products. "X" stands for out of GPU memory (49GB).
• Figure 4: The T-SNE plots of the condensed nodes. Left: SFGC(NeurIPS 2023) zheng2023structure, Right: DisCo.
• Figure 5: Evaluation of the proposed link prediction model. "Simple" refers to simple link prediction model.