Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Knowledge-Driven Federated Graph Learning on Model Heterogeneity

Zhengyu Wu, Guang Zeng, Huilin Lai, Daohan Su, Jishuo Jia, Yinlin Zhu, Xunkai Li, Rong-Hua Li, Guoren Wang, Chenghu Zhou

TL;DR

The paper tackles model-centric heterogeneous federated graph learning (MHtFGL), where clients deploy diverse GNN architectures, by introducing FedGKC, which uses a Copilot Model on each client and two mechanisms: Self-Mutual Knowledge Distillation (SMKD) to align local and Copilot knowledge with topology-aware signals, and Knowledge-Aware Model Aggregation (KAMA) to weight contributions by data volume and knowledge quality. SMKD enables bidirectional distillation and topology-informed neighborhood transfer, while KAMA computes weights from data volume, knowledge strength, and knowledge clarity to steer aggregation toward informative clients. The approach is evaluated on eight graph benchmarks, showing an average improvement of $3.88\%$ and up to $8.13\%$ over state-of-the-art baselines in MHtFGL settings, with strong generalization to homogeneous FGL. Theoretical results establish the stability of SMKD and demonstrate that KAMA provides gradient directions closer to the global optimum, supporting FedGKC as a robust, privacy-preserving solution for heterogeneous graph learning across organizations.

Abstract

Federated graph learning (FGL) has emerged as a promising paradigm for collaborative graph representation learning, enabling multiple parties to jointly train models while preserving data privacy. However, most existing approaches assume homogeneous client models and largely overlook the challenge of model-centric heterogeneous FGL (MHtFGL), which frequently arises in practice when organizations employ graph neural networks (GNNs) of different scales and architectures.Such architectural diversity not only undermines smooth server-side aggregation, which presupposes a unified representation space shared across clients' updates, but also further complicates the transfer and integration of structural knowledge across clients. To address this issue, we propose the Federated Graph Knowledge Collaboration (FedGKC) framework. FedGKC introduces a lightweight Copilot Model on each client to facilitate knowledge exchange while local architectures are heterogeneous across clients, and employs two complementary mechanisms: Client-side Self-Mutual Knowledge Distillation, which transfers effective knowledge between local and copilot models through bidirectional distillation with multi-view perturbation; and Server-side Knowledge-Aware Model Aggregation, which dynamically assigns aggregation weights based on knowledge provided by clients. Extensive experiments on eight benchmark datasets demonstrate that FedGKC achieves an average accuracy gain of 3.88% over baselines in MHtFGL scenarios, while maintaining excellent performance in homogeneous settings.

Knowledge-Driven Federated Graph Learning on Model Heterogeneity

TL;DR

The paper tackles model-centric heterogeneous federated graph learning (MHtFGL), where clients deploy diverse GNN architectures, by introducing FedGKC, which uses a Copilot Model on each client and two mechanisms: Self-Mutual Knowledge Distillation (SMKD) to align local and Copilot knowledge with topology-aware signals, and Knowledge-Aware Model Aggregation (KAMA) to weight contributions by data volume and knowledge quality. SMKD enables bidirectional distillation and topology-informed neighborhood transfer, while KAMA computes weights from data volume, knowledge strength, and knowledge clarity to steer aggregation toward informative clients. The approach is evaluated on eight graph benchmarks, showing an average improvement of and up to over state-of-the-art baselines in MHtFGL settings, with strong generalization to homogeneous FGL. Theoretical results establish the stability of SMKD and demonstrate that KAMA provides gradient directions closer to the global optimum, supporting FedGKC as a robust, privacy-preserving solution for heterogeneous graph learning across organizations.

Abstract

Federated graph learning (FGL) has emerged as a promising paradigm for collaborative graph representation learning, enabling multiple parties to jointly train models while preserving data privacy. However, most existing approaches assume homogeneous client models and largely overlook the challenge of model-centric heterogeneous FGL (MHtFGL), which frequently arises in practice when organizations employ graph neural networks (GNNs) of different scales and architectures.Such architectural diversity not only undermines smooth server-side aggregation, which presupposes a unified representation space shared across clients' updates, but also further complicates the transfer and integration of structural knowledge across clients. To address this issue, we propose the Federated Graph Knowledge Collaboration (FedGKC) framework. FedGKC introduces a lightweight Copilot Model on each client to facilitate knowledge exchange while local architectures are heterogeneous across clients, and employs two complementary mechanisms: Client-side Self-Mutual Knowledge Distillation, which transfers effective knowledge between local and copilot models through bidirectional distillation with multi-view perturbation; and Server-side Knowledge-Aware Model Aggregation, which dynamically assigns aggregation weights based on knowledge provided by clients. Extensive experiments on eight benchmark datasets demonstrate that FedGKC achieves an average accuracy gain of 3.88% over baselines in MHtFGL scenarios, while maintaining excellent performance in homogeneous settings.
Paper Structure (35 sections, 3 theorems, 24 equations, 4 figures, 6 tables, 1 algorithm)

This paper contains 35 sections, 3 theorems, 24 equations, 4 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Federated Graph Learning
  4. Model-Centric Heterogeneous FL
  5. Knowledge Distillation on Graph
  6. Methodology
  7. Notations and Problem Formulation
  8. Overview
  9. Self-Mutual Knowledge Distillation
  10. Mutual distillation addressing model heterogeneity
  11. Self-distillation mining model potential
  12. Knowledge-Aware Model Aggregation
  13. Complexity Analysis
  14. Theoretical Analysis
  15. Stability and Robustness of SMKD
  16. ...and 20 more sections

Key Result

Theorem 1

The squared discrepancy between the strong and weak perturbations, measured by the normalized embedding difference, is upper bounded by the ratio of the self-distillation loss to the average squared perturbation magnitude.

Figures (4)

  • Figure 1: Illustration of the conventional FGL, heterogeneous federated learning approaches, and our proposed FedGKC.
  • Figure 2: The visualized training pipeline of FedGKC. The Macro Federated Training depicts the FGL architecture in its essence, in which each client uploads its data volume: $w_k^{vol}$, trained copilot model parameters: $\theta_{c,k}$, and knowledge-realted weights: $w_k^{knowled}$, to secured server for model aggregation. On clients, heterogenous model architectures are implemented, which are visually separated by different shapes and colors. The right-hand side of the figure use the Isolated Client #4 showcases the training. Notably, to have a better visual illustration for readers, we use fewer nodes from the same subgraph dataset to increase its display clarity. All symbols in the figure are aligned with our methodological descriptions in Sec. \ref{['sec: Overview']}. Copilot Model Training and Local Model Training refer to Sec. \ref{['subsec: mutual distillation']}, and Server-side Knowledge-aware Model Aggregation refers to Sec. \ref{['sec: Knowledge-Aware Model Aggregation']}.
  • Figure 3: (a) Ablation experiments of SMKD and KAMA. (b) Coarse local–copilot Alignment vs. full FedGKC using different model Architectures on the copilot model.
  • Figure 4: Hyperparameters analysis: (a) Different combinations between parameters $\alpha$ and $\beta$. (b) Different $\lambda$ values.

Theorems & Definitions (4)

  • Definition 1: Lipschitz Constant
  • Theorem 1
  • Theorem 2
  • Theorem 3