Scale Invariance of Graph Neural Networks

TL;DR

The paper tackles two core problems in Graph Neural Networks: lack of theoretical grounding for invariant learning and the need for a single model that performs well on both homophilic and heterophilic graphs. It introduces scale invariance for graphs, proves it for GCN variants, and presents ScaleNet, a unified multi-scale architecture with an adaptive self-loop strategy. It demonstrates that scale-invariant transformations can replace expensive edge weights in digraph inception networks and shows an equivalence between Hermitian Laplacian methods and incidence-normalized GraphSAGE, achieving state-of-the-art results on four homophilic and two heterophilic datasets. Empirically ScaleNet also exhibits robustness to data imbalance, highlighting practical benefits for real-world graph learning. The work provides both theoretical insights and scalable, adaptable tools for unified graph learning, though it notes grid-search overhead as a limitation for hyperparameter tuning.

Abstract

We address two fundamental challenges in Graph Neural Networks (GNNs): (1) the lack of theoretical support for invariance learning, a critical property in image processing, and (2) the absence of a unified model capable of excelling on both homophilic and heterophilic graph datasets. To tackle these issues, we establish and prove scale invariance in graphs, extending this key property to graph learning, and validate it through experiments on real-world datasets. Leveraging directed multi-scaled graphs and an adaptive self-loop strategy, we propose ScaleNet, a unified network architecture that achieves state-of-the-art performance across four homophilic and two heterophilic benchmark datasets. Furthermore, we show that through graph transformation based on scale invariance, uniform weights can replace computationally expensive edge weights in digraph inception networks while maintaining or improving performance. For another popular GNN approach to digraphs, we demonstrate the equivalence between Hermitian Laplacian methods and GraphSAGE with incidence normalization. ScaleNet bridges the gap between homophilic and heterophilic graph learning, offering both theoretical insights into scale invariance and practical advancements in unified graph learning. Our implementation is publicly available at https://github.com/Qin87/ScaleNet/tree/Aug23.

Figures (5)

• Figure 1: An illustration of scaled ego-graphs. For directed graphs, the 1-depth in-edge ego-graph comprises nodes labeled "I" along with the center node and all in-edges between them, whereas the 1-depth out-edge ego-graph comprises nodes labeled O along with the center node and all out-edges between them. The four types of 1-depth $2^{nd}$-scaled ego-graphs are composed of nodes labeled "IO", "OI", "II", and "OO", with the center node and all corresponding $2^{nd}$-scaled edges between them.
• Figure 2: Schematic depiction of multi-layer ScaleNet with d input channels and h hidden channels. For layer-wise aggregation, the original graph is derived into two $1^{st}$-scaled and four $2^{nd}$-scaled graphs. Three AGG-B blocks determine input selection for COMB1, which uses either a jumping knowledge architecture or addition. COMB2 represents the fusion of all layers' outputs. (The blue blocks are optional, including self-loop operations, non-linear activation functions, dropout, and layer normalization.)
• Figure 3: Edge augmentation by stacking multi-scale graphs in Digraph Inception Model.
• Figure 4: DiG VS. RiG
• Figure 5: Random edge weights

Theorems & Definitions (10)

• Definition 1: In-Neighbour
• Definition 2: Out-Neighbour
• Definition 3
• Definition 4
• Definition 5
• Definition 6: Scaled Adjacency Matrix and Scaled Graph
• Definition 7
• proof
• proof
• proof