Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Practical Federated Learning without a Server

Akash Dhasade, Anne-Marie Kermarrec, Erick Lavoie, Johan Pouwelse, Rishi Sharma, Martijn de Vos

TL;DR

Federated learning typically relies on a central server to coordinate model updates, which incurs high infrastructure costs and potential single points of failure. Plexus eliminates the server by employing a decentralized peer sampler to select small training subsets and a bandwidth-aware aggregator within each subset to drive rounds. Across realistic WAN traces and datasets up to 1000 nodes, Plexus achieves 1.4–1.6× faster time-to-accuracy, 15.8–292× less communication, and 30.5–77.9× less training resources than strong decentralized baselines, while maintaining accuracy comparable to server-based FL. This demonstrates that decentralized coordination can deliver scalable, efficient FL-style training without centralized infrastructure, with practical impact for large, heterogeneous networks.

Abstract

Federated Learning (FL) enables end-user devices to collaboratively train ML models without sharing raw data, thereby preserving data privacy. In FL, a central parameter server coordinates the learning process by iteratively aggregating the trained models received from clients. Yet, deploying a central server is not always feasible due to hardware unavailability, infrastructure constraints, or operational costs. We present Plexus, a fully decentralized FL system for large networks that operates without the drawbacks originating from having a central server. Plexus distributes the responsibilities of model aggregation and sampling among participating nodes while avoiding network-wide coordination. We evaluate Plexus using realistic traces for compute speed, pairwise latency and network capacity. Our experiments on three common learning tasks and with up to 1000 nodes empirically show that Plexus reduces time-to-accuracy by 1.4-1.6x, communication volume by 15.8-292x and training resources needed for convergence by 30.5-77.9x compared to conventional decentralized learning algorithms.

Practical Federated Learning without a Server

TL;DR

Federated learning typically relies on a central server to coordinate model updates, which incurs high infrastructure costs and potential single points of failure. Plexus eliminates the server by employing a decentralized peer sampler to select small training subsets and a bandwidth-aware aggregator within each subset to drive rounds. Across realistic WAN traces and datasets up to 1000 nodes, Plexus achieves 1.4–1.6× faster time-to-accuracy, 15.8–292× less communication, and 30.5–77.9× less training resources than strong decentralized baselines, while maintaining accuracy comparable to server-based FL. This demonstrates that decentralized coordination can deliver scalable, efficient FL-style training without centralized infrastructure, with practical impact for large, heterogeneous networks.

Abstract

Federated Learning (FL) enables end-user devices to collaboratively train ML models without sharing raw data, thereby preserving data privacy. In FL, a central parameter server coordinates the learning process by iteratively aggregating the trained models received from clients. Yet, deploying a central server is not always feasible due to hardware unavailability, infrastructure constraints, or operational costs. We present Plexus, a fully decentralized FL system for large networks that operates without the drawbacks originating from having a central server. Plexus distributes the responsibilities of model aggregation and sampling among participating nodes while avoiding network-wide coordination. We evaluate Plexus using realistic traces for compute speed, pairwise latency and network capacity. Our experiments on three common learning tasks and with up to 1000 nodes empirically show that Plexus reduces time-to-accuracy by 1.4-1.6x, communication volume by 15.8-292x and training resources needed for convergence by 30.5-77.9x compared to conventional decentralized learning algorithms.

Paper Structure

This paper contains 15 sections, 5 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Motivation
  3. Design of Plexus
  4. System Model and Assumptions
  5. Plexus in a Nutshell
  6. Deriving Samples and Aggregators
  7. Training and Aggregating Models
  8. Plexus and FL
  9. Experimental Evaluation
  10. Experiment Setup
  11. Plexus Compared to Baselines
  12. Varying the Sample Size $s$
  13. Related Work
  14. Broader Impact and Open Challenges
  15. Conclusions

Figures (5)

  • Figure 1: The evolution of test accuracy of FL and DL (D-PSGD) on the CIFAR-10 dataset in a 1000.0-node network. FL converges quicker and to higher accuracy than DL.
  • Figure 2: Overview of round $k$ and $k+1$ in Plexus, with 8 nodes and a sample size of 4 ($s = 4$). Participants are indicated in blue and the aggregator in green.
  • Figure 3: The convergence of Plexus and baselines against time (top), communication volume (middle), and training resource usage (bottom).
  • Figure 4: The performance of Plexus for different sample sizes $s$, using the FEMNIST dataset.
  • Figure 5: The distribution of round durations, for the FEMNIST dataset and different sample sizes.