Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Federated Prototype Graph Learning

Zhengyu Wu, Xunkai Li, Yinlin Zhu, Rong-Hua Li, Guoren Wang, Chenghu Zhou

TL;DR

This work tackles multi-level heterogeneity in Federated Graph Learning by introducing FedPG, a prototype-guided optimization framework that shifts collaboration from model weights to topology-aware node embeddings. On the client side, FedPG builds topology-aware local prototypes across multiple Hop radii, while on the server side a trainable Global Prototype Generator creates universal prototypes guided by topology-aware contrastive learning, followed by personalized fusion for each client. Empirical results across 14 datasets demonstrate state-of-the-art accuracy with substantially lower communication costs and faster convergence, and ablation studies confirm the contributions of RF-context, margin-based enhancement, and prototype fusion. The approach shows robustness to privacy-preserving noise, scales to graphs of varying sizes, and generalizes well to Graph-FL setups, offering a practical baseline for prototype-based FGL research and deployment.

Abstract

In recent years, Federated Graph Learning (FGL) has gained significant attention for its distributed training capabilities in graph-based machine intelligence applications, mitigating data silos while offering a new perspective for privacy-preserve large-scale graph learning. However, multi-level FGL heterogeneity presents various client-server collaboration challenges: (1) Model-level: The variation in clients for expected performance and scalability necessitates the deployment of heterogeneous models. Unfortunately, most FGL methods rigidly demand identical client models due to the direct model weight aggregation on the server. (2) Data-level: The intricate nature of graphs, marked by the entanglement of node profiles and topology, poses an optimization dilemma. This implies that models obtained by federated training struggle to achieve superior performance. (3) Communication-level: Some FGL methods attempt to increase message sharing among clients or between clients and the server to improve training, which inevitably leads to high communication costs. In this paper, we propose FedPG as a general prototype-guided optimization method for the above multi-level FGL heterogeneity. Specifically, on the client side, we integrate multi-level topology-aware prototypes to capture local graph semantics. Subsequently, on the server side, leveraging the uploaded prototypes, we employ topology-guided contrastive learning and personalized technology to tailor global prototypes for each client, broadcasting them to improve local training. Experiments demonstrate that FedPG outperforms SOTA baselines by an average of 3.57\% in accuracy while reducing communication costs by 168x.

Federated Prototype Graph Learning

TL;DR

This work tackles multi-level heterogeneity in Federated Graph Learning by introducing FedPG, a prototype-guided optimization framework that shifts collaboration from model weights to topology-aware node embeddings. On the client side, FedPG builds topology-aware local prototypes across multiple Hop radii, while on the server side a trainable Global Prototype Generator creates universal prototypes guided by topology-aware contrastive learning, followed by personalized fusion for each client. Empirical results across 14 datasets demonstrate state-of-the-art accuracy with substantially lower communication costs and faster convergence, and ablation studies confirm the contributions of RF-context, margin-based enhancement, and prototype fusion. The approach shows robustness to privacy-preserving noise, scales to graphs of varying sizes, and generalizes well to Graph-FL setups, offering a practical baseline for prototype-based FGL research and deployment.

Abstract

In recent years, Federated Graph Learning (FGL) has gained significant attention for its distributed training capabilities in graph-based machine intelligence applications, mitigating data silos while offering a new perspective for privacy-preserve large-scale graph learning. However, multi-level FGL heterogeneity presents various client-server collaboration challenges: (1) Model-level: The variation in clients for expected performance and scalability necessitates the deployment of heterogeneous models. Unfortunately, most FGL methods rigidly demand identical client models due to the direct model weight aggregation on the server. (2) Data-level: The intricate nature of graphs, marked by the entanglement of node profiles and topology, poses an optimization dilemma. This implies that models obtained by federated training struggle to achieve superior performance. (3) Communication-level: Some FGL methods attempt to increase message sharing among clients or between clients and the server to improve training, which inevitably leads to high communication costs. In this paper, we propose FedPG as a general prototype-guided optimization method for the above multi-level FGL heterogeneity. Specifically, on the client side, we integrate multi-level topology-aware prototypes to capture local graph semantics. Subsequently, on the server side, leveraging the uploaded prototypes, we employ topology-guided contrastive learning and personalized technology to tailor global prototypes for each client, broadcasting them to improve local training. Experiments demonstrate that FedPG outperforms SOTA baselines by an average of 3.57\% in accuracy while reducing communication costs by 168x.

Paper Structure

This paper contains 17 sections, 7 equations, 7 figures, 10 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. FedPG Framework
  4. Client-side Local Prototype Representation
  5. Server-side Global Prototype Refinement
  6. Additional Optimization Details in $\operatorname{GPG}$
  7. FedPG Algorithm and Complexity Analysis
  8. Experiments
  9. Experimental Setup
  10. Experiment Environment
  11. Hyperparameter Settings
  12. Subgraph-FL Comparison
  13. Graph-FL Comparison.
  14. Ablation Study and In-depth Analysis
  15. Noise-based Privacy Preservation
  16. ...and 2 more sections

Figures (7)

  • Figure 1: A multi-level FGL heterogeneity illustration with the empirical case study.
  • Figure 2: Overview of our proposed FedPG framework.
  • Figure 3: Convergence curves with Louvain split and GCNII backbone.
  • Figure 4: Performance with RF prompts.
  • Figure 5: Sensitive analysis with Louvain split and GCN backbone.
  • ...and 2 more figures