Renormalized Graph Representations for Node Classification

TL;DR

This work addresses node classification by leveraging mesoscopic graph structures identified through the Laplacian Renormalization Group (LRG). It introduces a renormalization-based rewiring procedure that yields coarse-grained graphs at characteristic scales via diffusion-driven macro-nodes, and a multi-scale learning framework where GNN encoders process the original and renormalized graphs in parallel. A key finding is that incorporating representations from multiple scales—especially when including both the original and renormalized graphs (MR)—often yields statistically significant improvements in test accuracy, with the most effective scale identifiable a priori through spectral entropy: $C=-\frac{dS}{d(\log\tau)}$ peaks at the characteristic time $\tau^*$. The study demonstrates that the renormalized information provides complementary, scale-specific signals that enhance learning on several datasets, though benefits are dataset-dependent and depend on task characteristics. Overall, the work establishes RG-inspired multi-scale graph representations as a principled approach to inject long-range information into GNNs, with potential extensions to other graph learning tasks.

Abstract

Graph neural networks process information on graphs represented at a given resolution scale. We analyze the effect of using different coarse-grained graph resolutions, obtained through the Laplacian renormalization group theory, on node classification tasks. At the theory's core is grouping nodes connected by significant information flow at a given time scale. Representations of the graph at different scales encode interaction information at different ranges. We specifically experiment using representations at the characteristic scale of the graph's mesoscopic structures. We provide the models with the original graph and the graph represented at the characteristic resolution scale and compare them to models that can only access the original graph. Our results showed that models with access to both the original graph and the characteristic scale graph can achieve statistically significant improvements in test accuracy.

Figures (5)

• Figure 1: The heat capacity, defined as the derivative of spectral entropy with respect to the logarithm of time, is plotted as a function of time. The peak in this plot identifies the characteristic time scale. The values are related to the Cora graph.
• Figure 2: A visualization of the proposed rewiring procedure. The nodes of the original graph (left) are clustered in three macro-nodes (second from left). The graph is rewired in such a way that nodes in the same macro-node share the same incoming and outgoing edges (third from left). Finally, the links between nodes of the same macro-node are removed to sparsify the graph (right).
• Figure 3: Representations of the tested models: one that focuses on a single scale (i.e., a typical GNN encoder + classifier) is shown at the top, while a model that considers two graph scales is shown at the bottom. Note that the two input graphs in the model at the bottom represent the same phenomenon, but viewed at different scales.
• Figure 4: Test accuracy of the multiscale model with acces to the renormalized representations, "ours", in orange and its baseline that access only the original graph, "baseline", in blue over epochs for the Cora Dataset.
• Figure 5: Test accuracy of the model using the characteristic scale ('ours') in blue, the best randomly found scale ('best_random') in orange, and the worst randomly found scale ('worst_random') in green across the three intervals, varying by epochs. The values refer to the Cora dataset.