L2G-Net: Local to Global Spectral Graph Neural Networks via Cauchy Factorizations

TL;DR

A novel factorization of the GFT is introduced into operators acting on subgraphs, which are then combined via a sequence of Cauchy matrices to propose a new class of spectral GNNs, which is term L2G-Net (Local-to-Global Net).

Abstract

Despite their theoretical advantages, spectral methods based on the graph Fourier transform (GFT) are seldom used in graph neural networks (GNNs) due to the cost of computing the eigenbasis and the lack of vertex-domain locality in spectral representations. As a result, most GNNs rely on local approximations such as polynomial Laplacian filters or message passing, which limit their ability to model long-range dependencies. In this paper, we introduce a novel factorization of the GFT into operators acting on subgraphs, which are then combined via a sequence of Cauchy matrices. We use this factorization to propose a new class of spectral GNNs, which we term L2G-Net (Local-to-Global Net). Unlike existing spectral methods, which are either fully global (when they use the GFT) or local (when they use polynomial filters), L2G-Net operates by processing the spectral representations of subgraphs and then combining them via structured matrices. Our algorithm avoids full eigendecompositions, exploiting graph topology to construct the factorization with quadratic complexity in the number of nodes, scaled by the subgraph interface size. Experiments on benchmarks stressing non-local dependencies show that L2G-Net outperforms existing spectral techniques and is competitive with the state-of-the-art with orders of magnitude fewer learnable parameters.

Figures (5)

• Figure 1: (a) A graph in $\mathcal{F}(L=3, \lbrace \mathcal{G}_{i}\rbrace_{i = 1}^8, k=3)$. Bridge edges are shown in different colors. The GFT basis of the final graph can be expressed as the product of the stack of GFT bases of the base graphs by a sequence of Cauchy factors. (b) We first compute the base GFTs of small subgraphs (right) and apply local spectral filters. We mix the outputs via Cauchy factors corresponding to bridge edges. After all hierarchical merges, a global spectral filter acts on the full graph representation (left). This allows for global filtering that preserves locality while avoiding full eigendecompositions.
• Figure 2: Runtime on a random graph for eigendecomposition (ED), Cauchy factorization (CF), and preprocessing (SC+Sparse). The Cauchy factorization scales quadratically with the number of nodes. The preprocessing time is negligible.
• Figure 3: Runtime as a function of $k$ on a random graph for Cauchy factorization (CF) and preprocessing (SC+Sparse). The complexity of the Cauchy factorization scales linearly with the interface size. The preprocessing time is negligible and roughly constant with $k$.
• Figure 4: Performance vs complexity comparison. L2G-Net achieves competitive or superior performance with orders of magnitude fewer learnable parameters than state-of-the-art baselines, and outperforms the Global GFT across all datasets.
• Figure 5: Cumulative node contribution to the prediction on Minesweeper. Average across validation set; shaded areas indicate confidence intervals. L2G-Net localizes predictive power on a smaller fraction of nodes than Global GFT. The node contributions of Polynormer are less localized in the original graph.

Theorems & Definitions (25)

• Proposition 2.1: batson2014twice
• Definition 2.1: Cauchy matrix gastinel1960inversion
• Definition 2.2: Orthogonal Cauchy-like matrix, OCLM fasino2023orthogonalcai2018
• Definition 2.3: Hierarchical graph family (HGF)
• Definition 3.1: Cauchy factor (CF)
• Lemma 3.1: Progressive decomposition identity
• proof
• Theorem 3.1: Cauchy factorization
• proof
• Theorem 3.2