Learning Laplacian Positional Encodings for Heterophilous Graphs

TL;DR

This paper tackles the limitations of traditional Laplacian positional encodings (LPEs) in heterophilous graphs, where nearby nodes often have different labels. It introduces Learnable Laplacian Positional Encodings (LLPE), which leverage the full Laplacian spectrum and a learnable spectral filter implemented via Chebyshev polynomials to capture both homophily and heterophily. The authors prove LLPE’s expressivity to approximate a broad class of graph distances and establish favorable generalization properties, while empirically demonstrating improvements across 12 benchmarks, including up to 35% gains on synthetic data and 14% on real-world graphs, across diverse GNNs and graph transformers. They also propose scalable extensions using the Arnoldi method to handle large graphs. Overall, LLPE represents a principled, spectrum-aware approach to graph positional encodings with strong theoretical guarantees and practical impact for heterogeneous graph structures.

Abstract

In this work, we theoretically demonstrate that current graph positional encodings (PEs) are not beneficial and could potentially hurt performance in tasks involving heterophilous graphs, where nodes that are close tend to have different labels. This limitation is critical as many real-world networks exhibit heterophily, and even highly homophilous graphs can contain local regions of strong heterophily. To address this limitation, we propose Learnable Laplacian Positional Encodings (LLPE), a new PE that leverages the full spectrum of the graph Laplacian, enabling them to capture graph structure on both homophilous and heterophilous graphs. Theoretically, we prove LLPE's ability to approximate a general class of graph distances and demonstrate its generalization properties. Empirically, our evaluation on 12 benchmarks demonstrates that LLPE improves accuracy across a variety of GNNs, including graph transformers, by up to 35% and 14% on synthetic and real-world graphs, respectively. Going forward, our work represents a significant step towards developing PEs that effectively capture complex structures in heterophilous graphs.

Figures (9)

• Figure 1: LPEs on homophilous/heterophilous SBMs.
• Figure 2: LPEs and LLPEs on heterophilous SBMs.
• Figure 3: Mean and standard deviations (error bars) of all model-PE combinations on the synthetic SBMs. LLPE performs well across both high homophily and high heterophily, while LPE-FK does not.
• Figure 4: Performance of GTs with LPE-FK and LLPE across local node homophily quintiles on Cora and Amazon-ratings. Error bars are based on 95% bootstrapped confidence intervals.
• Figure 5: Mean and standard deviations (error bars) of all model-PE combinations on the synthetic SBMs. LLPE performs well across both high homophily and high heterophily, while LPE-FK does not.

Theorems & Definitions (31)

• Theorem 3.1: abbe2018community
• Theorem 3.2
• Proposition 4.1
• Definition 4.2
• Theorem 4.3
• Theorem 4.4
• Definition A.1
• Definition A.2
• Lemma A.3: stewart1990matrix
• Theorem A.4: yu2015useful, $\text{sin}\Theta$ theorem