Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Against Multifaceted Graph Heterogeneity via Asymmetric Federated Prompt Learning

Zhuoning Guo, Ruiqian Han, Hao Liu

TL;DR

A Federated Graph Prompt Learning (FedGPL) framework to efficiently enable prompt-based asymmetric graph knowledge transfer between multifaceted heterogeneous federated participants is proposed and two algorithms to eliminate task and data heterogeneity for advanced federated knowledge preservation are developed.

Abstract

Federated Graph Learning (FGL) aims to collaboratively and privately optimize graph models on divergent data for different tasks. A critical challenge in FGL is to enable effective yet efficient federated optimization against multifaceted graph heterogeneity to enhance mutual performance. However, existing FGL works primarily address graph data heterogeneity and perform incapable of graph task heterogeneity. To address the challenge, we propose a Federated Graph Prompt Learning (FedGPL) framework to efficiently enable prompt-based asymmetric graph knowledge transfer between multifaceted heterogeneous federated participants. Generally, we establish a split federated framework to preserve universal and domain-specific graph knowledge, respectively. Moreover, we develop two algorithms to eliminate task and data heterogeneity for advanced federated knowledge preservation. First, a Hierarchical Directed Transfer Aggregator (HiDTA) delivers cross-task beneficial knowledge that is hierarchically distilled according to the directional transferability. Second, a Virtual Prompt Graph (VPG) adaptively generates graph structures to enhance data utility by distinguishing dominant subgraphs and neutralizing redundant ones. We conduct theoretical analyses and extensive experiments to demonstrate the significant accuracy and efficiency effectiveness of FedGPL against multifaceted graph heterogeneity compared to state-of-the-art baselines on large-scale federated graph datasets.

Against Multifaceted Graph Heterogeneity via Asymmetric Federated Prompt Learning

TL;DR

A Federated Graph Prompt Learning (FedGPL) framework to efficiently enable prompt-based asymmetric graph knowledge transfer between multifaceted heterogeneous federated participants is proposed and two algorithms to eliminate task and data heterogeneity for advanced federated knowledge preservation are developed.

Abstract

Federated Graph Learning (FGL) aims to collaboratively and privately optimize graph models on divergent data for different tasks. A critical challenge in FGL is to enable effective yet efficient federated optimization against multifaceted graph heterogeneity to enhance mutual performance. However, existing FGL works primarily address graph data heterogeneity and perform incapable of graph task heterogeneity. To address the challenge, we propose a Federated Graph Prompt Learning (FedGPL) framework to efficiently enable prompt-based asymmetric graph knowledge transfer between multifaceted heterogeneous federated participants. Generally, we establish a split federated framework to preserve universal and domain-specific graph knowledge, respectively. Moreover, we develop two algorithms to eliminate task and data heterogeneity for advanced federated knowledge preservation. First, a Hierarchical Directed Transfer Aggregator (HiDTA) delivers cross-task beneficial knowledge that is hierarchically distilled according to the directional transferability. Second, a Virtual Prompt Graph (VPG) adaptively generates graph structures to enhance data utility by distinguishing dominant subgraphs and neutralizing redundant ones. We conduct theoretical analyses and extensive experiments to demonstrate the significant accuracy and efficiency effectiveness of FedGPL against multifaceted graph heterogeneity compared to state-of-the-art baselines on large-scale federated graph datasets.

Paper Structure

This paper contains 56 sections, 2 theorems, 10 equations, 6 figures, 11 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Notations and Problem Statement
  4. Graph notations.
  5. Multifaceted heterogeneity.
  6. Problem formulation.
  7. Threat model.
  8. Graph Prompt Learning
  9. Methodology
  10. Framework
  11. Algorithms
  12. Server-side federated aggregation.
  13. Client-side graph prompting.
  14. Training
  15. Theoretical Analysis
  16. ...and 41 more sections

Key Result

Theorem 1

For $a$ -th and $b$ -th clients in FedGPL, we denote their estimated optimized parameters at $l$-th step as $\theta_a^{(l+1)\prime},\theta_b^{(l+1)\prime}$ (aggregated) or $\theta_a^{(l)\prime},\theta_b^{(l)\prime}$ (non-aggregated). It is satisfied that when their bidirectional transferability are positive. Hence, the task heterogeneity can be reduced by HiDTA.

Figures (6)

  • Figure 1: An overview of Federated Graph Prompt Learning (FedGPL) framework.
  • Figure 2: Ablation study.
  • Figure 3: Graph inducing for three levels of tasks.
  • Figure 4: Performance-privacy correlation curves.
  • Figure 5: Transferability. N, E, and G: node-, edge-, and graph-level task, S: source task, and T: target task.
  • ...and 1 more figures

Theorems & Definitions (7)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Theorem 1
  • Theorem 2
  • Definition 5