Bi-Directional Multi-Scale Graph Dataset Condensation via Information Bottleneck

TL;DR

BiMSGC addresses efficient multi-scale graph dataset condensation for devices with varying compute budgets by introducing an IB-guided meso-scale subgraph and a bi-directional condensation objective, enhanced with eigenbasis matching. It formulates the condensation as maximizing $I( extbf{G}';H( extbf{G}, extbf{Y}))$ while selecting a meso-scale that preserves essential information through a Subgraph Condensation Information Bottleneck (SCIB) objective and scale transitions. The approach mitigates both scaling-down degradation and scaling-up collapse, achieving strong node-classification performance and robust cross-architecture generalization across five datasets, with notable speedups over baselines. This framework has practical impact for on-device graph learning by enabling flexible, information-preserving condensation across multiple resource profiles.

Abstract

Dataset condensation has significantly improved model training efficiency, but its application on devices with different computing power brings new requirements for different data sizes. Thus, condensing multiple scale graphs simultaneously is the core of achieving efficient training in different on-device scenarios. Existing efficient works for multi-scale graph dataset condensation mainly perform efficient approximate computation in scale order (large-to-small or small-to-large scales). However, for non-Euclidean structures of sparse graph data, these two commonly used paradigms for multi-scale graph dataset condensation have serious scaling down degradation and scaling up collapse problems of a graph. The main bottleneck of the above paradigms is whether the effective information of the original graph is fully preserved when consenting to the primary sub-scale (the first of multiple scales), which determines the condensation effect and consistency of all scales. In this paper, we proposed a novel GNN-centric Bi-directional Multi-Scale Graph Dataset Condensation (BiMSGC) framework, to explore unifying paradigms by operating on both large-to-small and small-to-large for multi-scale graph condensation. Based on the mutual information theory, we estimate an optimal ``meso-scale'' to obtain the minimum necessary dense graph preserving the maximum utility information of the original graph, and then we achieve stable and consistent ``bi-directional'' condensation learning by optimizing graph eigenbasis matching with information bottleneck on other scales. Encouraging empirical results on several datasets demonstrates the significant superiority of the proposed framework in graph condensation at different scales.

Figures (5)

• Figure 1: Comparison of existing paradigms
• Figure 2: (a) Three different paradigm-specific training strategies: Large-to-Small is trained by mask sampling from a large graph to obtain a small graph. Small-to-Large is trained by considering the small graph as a subgraph expansion of the large graph. Our method obtains meso-scale subgraphs and then trains them separately to both sides; (b) Condensation performance on each scale (top) and time efficiency (bottom) obtained by training four different multi-scale compression strategies on Ogbn-arxiv using GCond as a backbone method; (c) The magnitude of mutual information between the different target condensation scales of the three paradigms and the original graph (top), as well as the magnitude of mutual information with the largest scale condensed graph (bottom), where S-to-L represents Small-to-Large and L-to-S represents Large-to-Small.
• Figure 3: An illustration of BiMSGC architecture.
• Figure 4: Ablation study for IB with different meso-scales.
• Figure 5: Sensitivity study on meso-scale selection.