ScaleNet: Scale Invariance Learning in Directed Graphs

TL;DR

This research extends the scale invariance concept to node classification by drawing an analogy to image processing: just as scale invariance being used in image classification to capture multi-scale features, the concept of ``scaled ego-graphs''.

Abstract

Graph Neural Networks (GNNs) have advanced relational data analysis but lack invariance learning techniques common in image classification. In node classification with GNNs, it is actually the ego-graph of the center node that is classified. This research extends the scale invariance concept to node classification by drawing an analogy to image processing: just as scale invariance being used in image classification to capture multi-scale features, we propose the concept of ``scaled ego-graphs''. Scaled ego-graphs generalize traditional ego-graphs by replacing undirected single-edges with ``scaled-edges'', which are ordered sequences of multiple directed edges. We empirically assess the performance of the proposed scale invariance in graphs on seven benchmark datasets, across both homophilic and heterophilic structures. Our scale-invariance-based graph learning outperforms inception models derived from random walks by being simpler, faster, and more accurate. The scale invariance explains inception models' success on homophilic graphs and limitations on heterophilic graphs. To ensure applicability of inception model to heterophilic graphs as well, we further present ScaleNet, an architecture that leverages multi-scaled features. ScaleNet achieves state-of-the-art results on five out of seven datasets (four homophilic and one heterophilic) and matches top performance on the remaining two, demonstrating its excellent applicability. This represents a significant advance in graph learning, offering a unified framework that enhances node classification across various graph types. Our code is available at https://github.com/Qin87/ScaleNet/tree/July25.

Figures (6)

• Figure 1: An illustration of scaled ego-graphs. For directed graphs, the 1-depth in-edge ego-graph comprises nodes labeled "I" along with the center node and all in-edges between them, whereas the 1-depth out-edge ego-graph comprises nodes labeled O along with the center node and all out-edges between them. The four types of 1-depth $2^{nd}$-scaled ego-graphs are composed of nodes labeled "IO", "OI", "II", and "OO", with the center node and all corresponding $2^{nd}$-scaled edges between them.
• Figure 2: Edge augmentation by stacking multi-scale graphs in Digraph Inception Model. Details are provided in Appendix \ref{['DiG<RiG']}.
• Figure 3: Schematic depiction of multi-layer ScaleNet with d input channels and h hidden channels. For layer-wise aggregation, the original graph is derived into two $1^{st}$-scaled and four $2^{nd}$-scaled graphs. Three AGG-B blocks determine input selection for COMB1, which uses either a jumping knowledge architecture or addition. COMB2 represents the fusion of all layers' outputs. (The blue blocks are optional, including self-loop operations, non-linear activation functions, dropout, and layer normalization.)
• Figure 4: Edge augmentation by stacking multi-scale graphs.
• Figure 5: DiG VS. RiG

Theorems & Definitions (9)

• Definition 1: In-Neighbour
• Definition 2: Out-Neighbour
• Definition 3
• Definition 4
• Definition 5
• Definition 6: Scaled Adjacency Matrix and Scaled Graph
• Definition 7
• Proposition 1
• proof