Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FedML: A Research Library and Benchmark for Federated Machine Learning

Chaoyang He, Songze Li, Jinhyun So, Xiao Zeng, Mi Zhang, Hongyi Wang, Xiaoyang Wang, Praneeth Vepakomma, Abhishek Singh, Hang Qiu, Xinghua Zhu, Jianzong Wang, Li Shen, Peilin Zhao, Yan Kang, Yang Liu, Ramesh Raskar, Qiang Yang, Murali Annavaram, Salman Avestimehr

TL;DR

<FedML addresses core limitations in federated learning tooling by providing an open, end-to-end library and benchmark that support on-device, distributed, and single-machine FL. The architecture cleanly separates low-level communication from high-level algorithm design, using a worker-oriented programming model, flexible topology management, and extensible privacy/security features. The benchmark enforces standardized datasets, models, and non-I.I.D. partitions across convex, shallow non-convex, and deep non-convex tasks, with reference algorithm implementations and real-world on-device testbeds. This framework enables fair comparisons, reproducible experiments, and rapid experimentation, promoting broader, more rigorous FL research and deployment readiness.

Abstract

Federated learning (FL) is a rapidly growing research field in machine learning. However, existing FL libraries cannot adequately support diverse algorithmic development; inconsistent dataset and model usage make fair algorithm comparison challenging. In this work, we introduce FedML, an open research library and benchmark to facilitate FL algorithm development and fair performance comparison. FedML supports three computing paradigms: on-device training for edge devices, distributed computing, and single-machine simulation. FedML also promotes diverse algorithmic research with flexible and generic API design and comprehensive reference baseline implementations (optimizer, models, and datasets). We hope FedML could provide an efficient and reproducible means for developing and evaluating FL algorithms that would benefit the FL research community. We maintain the source code, documents, and user community at https://fedml.ai.

FedML: A Research Library and Benchmark for Federated Machine Learning

TL;DR

<FedML addresses core limitations in federated learning tooling by providing an open, end-to-end library and benchmark that support on-device, distributed, and single-machine FL. The architecture cleanly separates low-level communication from high-level algorithm design, using a worker-oriented programming model, flexible topology management, and extensible privacy/security features. The benchmark enforces standardized datasets, models, and non-I.I.D. partitions across convex, shallow non-convex, and deep non-convex tasks, with reference algorithm implementations and real-world on-device testbeds. This framework enables fair comparisons, reproducible experiments, and rapid experimentation, promoting broader, more rigorous FL research and deployment readiness.

Abstract

Federated learning (FL) is a rapidly growing research field in machine learning. However, existing FL libraries cannot adequately support diverse algorithmic development; inconsistent dataset and model usage make fair algorithm comparison challenging. In this work, we introduce FedML, an open research library and benchmark to facilitate FL algorithm development and fair performance comparison. FedML supports three computing paradigms: on-device training for edge devices, distributed computing, and single-machine simulation. FedML also promotes diverse algorithmic research with flexible and generic API design and comprehensive reference baseline implementations (optimizer, models, and datasets). We hope FedML could provide an efficient and reproducible means for developing and evaluating FL algorithms that would benefit the FL research community. We maintain the source code, documents, and user community at https://fedml.ai.

Paper Structure

This paper contains 27 sections, 5 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Lack of support of diverse FL computing paradigms.
  3. Lack of support of diverse FL configurations.
  4. Lack of standardized FL algorithm implementations and benchmarks.
  5. FedML Library: Architecture Design
  6. FedML Library: Programming Interface
  7. Worker/client-oriented programming.
  8. Message definition beyond gradient and model.
  9. Topology management.
  10. Trainer and coordinator.
  11. Privacy, security, and robustness.
  12. FedML Benchmark: Algorithms, Models, and Datasets
  13. Algorithms: Federated Optimizer
  14. Models and Datasets
  15. Federated datasets for linear models (convex optimization).
  16. ...and 12 more sections

Figures (5)

  • Figure 1: Overview of FedML library.
  • Figure 2: A worker-oriented programming design pattern of FedML.
  • Figure 3: Illustration of various topology definitions in federated learning.
  • Figure 4: Supported algorithms that are diverse in network topology, exchanged information, and training procedures.
  • Figure 5: Test accuracy of ResNet-56 during training.