Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Personalized Federated Learning via Learning Dynamic Graphs

Ziran Zhou, Guanyu Gao, Xiaohu Wu, Yan Lyu

TL;DR

This work addresses personalized federated learning under non-IID data by shifting focus from merely personalizing local models to learning how to aggregate models across clients. It introduces pFedGAT, a server-side Graph Attention Network that starts from a fully connected client graph and dynamically computes per-client aggregation weights, producing personalized updates via $\theta_i^{t+1} = \sum_j R_{ij}^t \theta_j^t$ where $R^t$ is the multi-head attention-derived weight matrix. An end-to-end optimization framework uses client-side test feedback to update the GAT parameters $W$ and $a$ with backpropagation, achieving adaptation with minimal communication overhead. Empirical results on Fashion-MNIST, CIFAR-10, and CIFAR-100 across varied data heterogeneity and client counts show pFedGAT consistently outperforms twelve baselines, demonstrating robustness, effective graph-aware aggregation, and good generalization to new clients.

Abstract

Personalized Federated Learning (PFL) aims to train a personalized model for each client that is tailored to its local data distribution, learning fails to perform well on individual clients due to variations in their local data distributions. Most existing PFL methods focus on personalizing the aggregated global model for each client, neglecting the fundamental aspect of federated learning: the regulation of how client models are aggregated. Additionally, almost all of them overlook the graph structure formed by clients in federated learning. In this paper, we propose a novel method, Personalized Federated Learning with Graph Attention Network (pFedGAT), which captures the latent graph structure between clients and dynamically determines the importance of other clients for each client, enabling fine-grained control over the aggregation process. We evaluate pFedGAT across multiple data distribution scenarios, comparing it with twelve state of the art methods on three datasets: Fashion MNIST, CIFAR-10, and CIFAR-100, and find that it consistently performs well.

Personalized Federated Learning via Learning Dynamic Graphs

TL;DR

This work addresses personalized federated learning under non-IID data by shifting focus from merely personalizing local models to learning how to aggregate models across clients. It introduces pFedGAT, a server-side Graph Attention Network that starts from a fully connected client graph and dynamically computes per-client aggregation weights, producing personalized updates via where is the multi-head attention-derived weight matrix. An end-to-end optimization framework uses client-side test feedback to update the GAT parameters and with backpropagation, achieving adaptation with minimal communication overhead. Empirical results on Fashion-MNIST, CIFAR-10, and CIFAR-100 across varied data heterogeneity and client counts show pFedGAT consistently outperforms twelve baselines, demonstrating robustness, effective graph-aware aggregation, and good generalization to new clients.

Abstract

Personalized Federated Learning (PFL) aims to train a personalized model for each client that is tailored to its local data distribution, learning fails to perform well on individual clients due to variations in their local data distributions. Most existing PFL methods focus on personalizing the aggregated global model for each client, neglecting the fundamental aspect of federated learning: the regulation of how client models are aggregated. Additionally, almost all of them overlook the graph structure formed by clients in federated learning. In this paper, we propose a novel method, Personalized Federated Learning with Graph Attention Network (pFedGAT), which captures the latent graph structure between clients and dynamically determines the importance of other clients for each client, enabling fine-grained control over the aggregation process. We evaluate pFedGAT across multiple data distribution scenarios, comparing it with twelve state of the art methods on three datasets: Fashion MNIST, CIFAR-10, and CIFAR-100, and find that it consistently performs well.

Paper Structure

This paper contains 21 sections, 10 equations, 4 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Federated Learning
  4. Personalized Federated Learning
  5. Graph Learning
  6. Method
  7. Problem Formulation
  8. pFedGAT Algorithm
  9. Graph Construction
  10. Attention-Based Weight Assignment
  11. End-to-End Optimization with Client Feedback
  12. Experiments
  13. Experiments Setup
  14. Main Experiment
  15. Different Degrees of Heterogeneity
  16. ...and 6 more sections

Figures (4)

  • Figure 1: Workflow of pFedGAT
  • Figure 2: Visualization of Data Distributions Across Clients on CIFAR-10 Under Different Heterogeneity Levels
  • Figure 3: Visualization of the weight allocation matrices under IID and Pathological data distribution.
  • Figure 4: A Comparison of Generalization Performance on CIFAR-10 and CIFAR-100. Our method consistently achieves the highest performance across all scenarios.