Federated Spectral Graph Transformers Meet Neural Ordinary Differential Equations for Non-IID Graphs

TL;DR

This paper tackles privacy-preserving learning on graph-structured data by proposing GNODEFormer, a spectral GNN that integrates neural ordinary differential equations to model continuous feature propagation, and its federated extension Fed-GNODEFormer designed to handle non-IID, heterophilic graphs. Central to GNODEFormer is the eigen-decomposition of the normalized graph Laplacian $\mathcal{L} = \mathcal{I}_n - \mathcal{D}^{-1/2} \mathcal{A} \mathcal{D}^{-1/2}$ with $\mathcal{L} = \mathcal{U} \Psi \mathcal{U}^T$, and an eigenvalue encoding that feeds high-order ODE transformer blocks. The method combines a decoding step that reconstructs a refined Laplacian $\mathcal{L}_{new} = \mathcal{U} \mathrm{diag}(\gamma_n) \mathcal{U}^T$ with spectral graph convolution to produce robust node representations, and extends to FL via Dirichlet-partitioned subgraphs aggregated by FedAvg. Empirical results demonstrate state-of-the-art performance on heterophilic datasets and competitive results on IID/homophilic graphs, with RK-4 offering higher accuracy at increased computational cost. The work advances practical federated graph learning by leveraging spectral methods and continuous dynamics to address non-IID settings in real-world networks.

Abstract

Graph Neural Network (GNN) research is rapidly advancing due to GNNs' capacity to learn distributed representations from graph-structured data. However, centralizing large volumes of real-world graph data for GNN training is often impractical due to privacy concerns, regulatory restrictions, and commercial competition. Federated learning (FL), a distributed learning paradigm, offers a solution by preserving data privacy with collaborative model training. Despite progress in training huge vision and language models, federated learning for GNNs remains underexplored. To address this challenge, we present a novel method for federated learning on GNNs based on spectral GNNs equipped with neural ordinary differential equations (ODE) for better information capture, showing promising results across both homophilic and heterophilic graphs. Our approach effectively handles non-Independent and Identically Distributed (non-IID) data, while also achieving performance comparable to existing methods that only operate on IID data. It is designed to be privacy-preserving and bandwidth-optimized, making it suitable for real-world applications such as social network analysis, recommendation systems, and fraud detection, which often involve complex, non-IID, and heterophilic graph structures. Our results in the area of federated learning on non-IID heterophilic graphs demonstrate significant improvements, while also achieving better performance on homophilic graphs. This work highlights the potential of federated learning in diverse and challenging graph settings. Open-source code available on GitHub (https://github.com/SpringWiz11/Fed-GNODEFormer).

Figures (15)

• Figure 1: GNODEFormer architecture: Integrating ODE-inspired Runge-Kutta methods, Residual Layer History, and spectral graph convolution for enhanced graph representation learning.
• Figure 2: Fed-GNODEFormer: GNODEFormer in a Federated Learning setup. The diagram illustrates multiple clients, each with a local model, interacting with a central server. The numbered arrows represent the standard FL workflow: (1) the server sends the global parameters, (2) clients perform local updates, (3) clients send the updated parameters back, and (4) the server aggregates them. Arrows indicate the flow of parameters and updates for explanation purposes and do not imply a directed graph structure.
• Figure 3: Comparison of Model Performance on chameleon. The table (left) shows the average accuracy and standard deviation of the model across different $\alpha$ values for rk-2 model. The adjacent graph (right) provides a visual representation that further illustrates the trend in model performance, offering a clearer understanding of how the model adapts to varying data distributions.
• Figure 5: Eigenvalue plots for different datasets.
• Figure 6: