Lumos: Heterogeneity-aware Federated Graph Learning over Decentralized Devices

TL;DR

Lumos tackles privacy-preserving node-level federated graph learning by protecting both local features and node degrees while addressing workload heterogeneity across devices. It introduces a heterogeneity-aware tree constructor to generate expressive, balanced tree-structured ego-networks and a decentralized tree-based GNN trainer that performs privacy-preserving embedding learning with local DP for features and zero-knowledge degree comparisons. A Monte Carlo Markov Chain–based trimming algorithm balances workload and provides theoretical guarantees on convergence, enabling efficient, scalable training. Empirical results on real-world graphs show Lumos approaches centralized GNN performance while substantially reducing communication rounds and training time, demonstrating practical impact for decentralized and privacy-conscious graph analytics.

Abstract

Graph neural networks (GNN) have been widely deployed in real-world networked applications and systems due to their capability to handle graph-structured data. However, the growing awareness of data privacy severely challenges the traditional centralized model training paradigm, where a server holds all the graph information. Federated learning is an emerging collaborative computing paradigm that allows model training without data centralization. Existing federated GNN studies mainly focus on systems where clients hold distinctive graphs or sub-graphs. The practical node-level federated situation, where each client is only aware of its direct neighbors, has yet to be studied. In this paper, we propose the first federated GNN framework called Lumos that supports supervised and unsupervised learning with feature and degree protection on node-level federated graphs. We first design a tree constructor to improve the representation capability given the limited structural information. We further present a Monte Carlo Markov Chain-based algorithm to mitigate the workload imbalance caused by degree heterogeneity with theoretically-guaranteed performance. Based on the constructed tree for each client, a decentralized tree-based GNN trainer is proposed to support versatile training. Extensive experiments demonstrate that Lumos outperforms the baseline with significantly higher accuracy and greatly reduced communication cost and training time.

Figures (8)

• Figure 1: Lumos overview: unique blocks (greedy initialization, MCMC iteration and POOL layer) represent parts involving inter-device communication and duplicated blocks (embedding initialization and GNN) represent parts operated locally.
• Figure 2: Constructed tree for a given ego network
• Figure 3: Label classification accuracy
• Figure 4: Link prediction ROC-AUC score
• Figure 5: Effect of privacy parameter $\epsilon$

Theorems & Definitions (13)

• Definition 1
• Definition 2
• Definition 3: Degree Heterogeneity
• Theorem 1
• proof
• Theorem 2
• proof
• Theorem 3
• proof
• Theorem 4