Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Subgraph Federated Learning via Spectral Methods

Javad Aliakbari, Johan Östman, Ashkan Panahi, Alexandre Graell i Amat

TL;DR

This work tackles subgraph federated learning for node classification on globally connected graphs by introducing FedLap, a framework that uses Laplacian smoothing in the spectral domain to exploit global structure while preserving privacy and scalability. It advances FedLap+ by decomposing the structural matrix into a fixed spectral basis and a learnable low-rank component, enabling efficient, privacy-preserving spectral information exchange via a decentralized Arnoldi iteration. A formal privacy analysis under a strong attacker model establishes strong guarantees for FedLap+, distinguishing it from prior SFL methods lacking such analysis. Empirical results across six benchmarks demonstrate competitive accuracy with lower communication and enhanced privacy, highlighting the practicality of spectral domain SFL for large-scale graphs.

Abstract

We consider the problem of federated learning (FL) with graph-structured data distributed across multiple clients. In particular, we address the prevalent scenario of interconnected subgraphs, where interconnections between clients significantly influence the learning process. Existing approaches suffer from critical limitations, either requiring the exchange of sensitive node embeddings, thereby posing privacy risks, or relying on computationally-intensive steps, which hinders scalability. To tackle these challenges, we propose FedLap, a novel framework that leverages global structure information via Laplacian smoothing in the spectral domain to effectively capture inter-node dependencies while ensuring privacy and scalability. We provide a formal analysis of the privacy of FedLap, demonstrating that it preserves privacy. Notably, FedLap is the first subgraph FL scheme with strong privacy guarantees. Extensive experiments on benchmark datasets demonstrate that FedLap achieves competitive or superior utility compared to existing techniques.

Subgraph Federated Learning via Spectral Methods

TL;DR

This work tackles subgraph federated learning for node classification on globally connected graphs by introducing FedLap, a framework that uses Laplacian smoothing in the spectral domain to exploit global structure while preserving privacy and scalability. It advances FedLap+ by decomposing the structural matrix into a fixed spectral basis and a learnable low-rank component, enabling efficient, privacy-preserving spectral information exchange via a decentralized Arnoldi iteration. A formal privacy analysis under a strong attacker model establishes strong guarantees for FedLap+, distinguishing it from prior SFL methods lacking such analysis. Empirical results across six benchmarks demonstrate competitive accuracy with lower communication and enhanced privacy, highlighting the practicality of spectral domain SFL for large-scale graphs.

Abstract

We consider the problem of federated learning (FL) with graph-structured data distributed across multiple clients. In particular, we address the prevalent scenario of interconnected subgraphs, where interconnections between clients significantly influence the learning process. Existing approaches suffer from critical limitations, either requiring the exchange of sensitive node embeddings, thereby posing privacy risks, or relying on computationally-intensive steps, which hinders scalability. To tackle these challenges, we propose FedLap, a novel framework that leverages global structure information via Laplacian smoothing in the spectral domain to effectively capture inter-node dependencies while ensuring privacy and scalability. We provide a formal analysis of the privacy of FedLap, demonstrating that it preserves privacy. Notably, FedLap is the first subgraph FL scheme with strong privacy guarantees. Extensive experiments on benchmark datasets demonstrate that FedLap achieves competitive or superior utility compared to existing techniques.

Paper Structure

This paper contains 41 sections, 4 theorems, 68 equations, 9 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries and Setup
  4. FedLap
  5. FedLap: Exploiting Structural Information in SFL via Laplacian Smoothing
  6. FedLap+: Exploiting Structural Information in the Spectral Domain
  7. Decentralized Arnoldi Iteration: Privacy-Preserving Approximation of the Laplacian
  8. Privacy Analysis of FedLap+
  9. Attacker observations and procedure.
  10. Communication Complexity
  11. Experimental Results
  12. Prediction model with FedStruct
  13. Details of the Arnoldi Iteration Method
  14. Standard Arnoldi Iteration
  15. Proposed Decentralized Arnoldi Iteration
  16. ...and 26 more sections

Key Result

Theorem 1

Consider two clients running the decentralized Arnoldi scheme outlined in Sec. ss:arnoldi_dec. Let $\bm{A}$ be a random graph with $p$ denoting the probability of a connection between any pair $(u,v)$ for $u, v \in \mathcal{V}_1$. Assume $p$ to be known by client $2$. Let $\bm{U}=\boldsymbol{\breve{ is a random variable with the distribution where $\alpha_v = \boldsymbol{\breve{Q}}_{v,:} \bm{\Sig

Figures (9)

  • Figure 1: FedLap+ with three clients. Left: the global graph and its partitioning across clients. Center: local refinement of the global eigenvectors obtained via Arnoldi iterations; the corresponding adjacency matrix is shown below. Right: federated learning leveraging the estimated global eigenvectors.
  • Figure 1: Communication cost.
  • Figure 2: Comparison of different matrix representations in the graph. In (b) and (c), $r = 100$ for dimensionality reduction.
  • Figure 3: Effect of the rank parameter $r$ on the precision + recall with varying $\gamma$ for the Chameleon (left), Amazon photo (center), and PubMed (right) datasets. The pairs $(p,n)$ are $(0.0139,2277)$, $(0.008,7650)$, and $(0.0005,19717)$, respectively. The curves illustrate how the choice of $r$ impacts the trade-off between recall and precision as the decision threshold varies.
  • Figure 4: Precision vs recall on PubMed.
  • ...and 4 more figures

Theorems & Definitions (8)

  • Theorem 1
  • proof
  • Corollary 1
  • proof
  • Lemma C.1
  • proof
  • Lemma C.2: Berry--Esseen bound for Bernoulli graph models
  • proof : Proof