Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Version age-based client scheduling policy for federated learning

Xinyi Hu, Nikolaos Pappas, Howard H. Yang

TL;DR

We address straggler-driven instability in federated learning by introducing Version Age of Information (VAoI), a discrete per-client freshness metric that blends timeliness and content staleness. The method defines per-client VAoI $X_i(t)$ and uses a nonlinear aging cost $h(x)$, e.g., $h(x)=\exp(x)$, to set selection probabilities $p_i(t)$, biasing client sampling toward stale updates. Experiments on CIFAR-100 with ResNet-18 and non-IID Dirichlet data show that VAoI-based scheduling improves test accuracy and reduces average VAoI compared to FedAvg, with average VAoI peaking near $2.8$ around round $275$ and then decreasing. The work demonstrates that VAoI is a practical metric for robustness in bandwidth-constrained FL and motivates further theoretical convergence analysis for the proposed scheduling policy.

Abstract

Federated Learning (FL) has emerged as a privacy-preserving machine learning paradigm facilitating collaborative training across multiple clients without sharing local data. Despite advancements in edge device capabilities, communication bottlenecks present challenges in aggregating a large number of clients; only a portion of the clients can update their parameters upon each global aggregation. This phenomenon introduces the critical challenge of stragglers in FL and the profound impact of client scheduling policies on global model convergence and stability. Existing scheduling strategies address staleness but predominantly focus on either timeliness or content. Motivated by this, we introduce the novel concept of Version Age of Information (VAoI) to FL. Unlike traditional Age of Information metrics, VAoI considers both timeliness and content staleness. Each client's version age is updated discretely, indicating the freshness of information. VAoI is incorporated into the client scheduling policy to minimize the average VAoI, mitigating the impact of outdated local updates and enhancing the stability of FL systems.

Version age-based client scheduling policy for federated learning

TL;DR

We address straggler-driven instability in federated learning by introducing Version Age of Information (VAoI), a discrete per-client freshness metric that blends timeliness and content staleness. The method defines per-client VAoI and uses a nonlinear aging cost , e.g., , to set selection probabilities , biasing client sampling toward stale updates. Experiments on CIFAR-100 with ResNet-18 and non-IID Dirichlet data show that VAoI-based scheduling improves test accuracy and reduces average VAoI compared to FedAvg, with average VAoI peaking near around round and then decreasing. The work demonstrates that VAoI is a practical metric for robustness in bandwidth-constrained FL and motivates further theoretical convergence analysis for the proposed scheduling policy.

Abstract

Federated Learning (FL) has emerged as a privacy-preserving machine learning paradigm facilitating collaborative training across multiple clients without sharing local data. Despite advancements in edge device capabilities, communication bottlenecks present challenges in aggregating a large number of clients; only a portion of the clients can update their parameters upon each global aggregation. This phenomenon introduces the critical challenge of stragglers in FL and the profound impact of client scheduling policies on global model convergence and stability. Existing scheduling strategies address staleness but predominantly focus on either timeliness or content. Motivated by this, we introduce the novel concept of Version Age of Information (VAoI) to FL. Unlike traditional Age of Information metrics, VAoI considers both timeliness and content staleness. Each client's version age is updated discretely, indicating the freshness of information. VAoI is incorporated into the client scheduling policy to minimize the average VAoI, mitigating the impact of outdated local updates and enhancing the stability of FL systems.
Paper Structure (8 sections, 6 equations, 4 figures, 2 algorithms)

This paper contains 8 sections, 6 equations, 4 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. System model
  3. Federated Learning
  4. Version Age of Information
  5. Version Age-based Scheduling Policy
  6. Algorithm
  7. Numerical Results
  8. Conclusion

Figures (4)

  • Figure 1: An example of the version age evolution over time, the upward arrows represent updates received by the client.
  • Figure 2: Overview of proposed scheduling policy.
  • Figure 3: Impact of linear and non-linear $h(\cdot).$
  • Figure 4: Impact of scheduling policy on the convergence of FL in terms of test accuracy and average version age.