Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Energy-efficient Federated Learning with Dynamic Model Size Allocation

M S Chaitanya Kumar, Sai Satya Narayana J, Yunkai Bao, Xin Wang, Steve Drew

TL;DR

CAMA is proposed, a carbon-aware FL framework, promoting the operation on renewable excess energy and spare computing capacity, aiming to minimize operational carbon emissions and achieves faster convergence and ensures equitable client participation, while scaling efficiently to handle large numbers of clients.

Abstract

Federated Learning (FL) presents a paradigm shift towards distributed model training across isolated data repositories or edge devices without explicit data sharing. Despite of its advantages, FL is inherently less efficient than centralized training models, leading to increased energy consumption and, consequently, higher carbon emissions. In this paper, we propose CAMA, a carbon-aware FL framework, promoting the operation on renewable excess energy and spare computing capacity, aiming to minimize operational carbon emissions. CAMA introduces a dynamic model adaptation strategy which adapts the model sizes based on the availability of energy and computing resources. Ordered dropout is integratged to enable the aggregation with varying model sizes. Empirical evaluations on real-world energy and load traces demonstrate that our method achieves faster convergence and ensures equitable client participation, while scaling efficiently to handle large numbers of clients. The source code of CAMA is available at https://github.com/denoslab/CAMA.

Energy-efficient Federated Learning with Dynamic Model Size Allocation

TL;DR

CAMA is proposed, a carbon-aware FL framework, promoting the operation on renewable excess energy and spare computing capacity, aiming to minimize operational carbon emissions and achieves faster convergence and ensures equitable client participation, while scaling efficiently to handle large numbers of clients.

Abstract

Federated Learning (FL) presents a paradigm shift towards distributed model training across isolated data repositories or edge devices without explicit data sharing. Despite of its advantages, FL is inherently less efficient than centralized training models, leading to increased energy consumption and, consequently, higher carbon emissions. In this paper, we propose CAMA, a carbon-aware FL framework, promoting the operation on renewable excess energy and spare computing capacity, aiming to minimize operational carbon emissions. CAMA introduces a dynamic model adaptation strategy which adapts the model sizes based on the availability of energy and computing resources. Ordered dropout is integratged to enable the aggregation with varying model sizes. Empirical evaluations on real-world energy and load traces demonstrate that our method achieves faster convergence and ensures equitable client participation, while scaling efficiently to handle large numbers of clients. The source code of CAMA is available at https://github.com/denoslab/CAMA.

Paper Structure

This paper contains 10 sections, 3 equations, 4 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Problem Statement
  3. Client Selection Strategy
  4. Client Registration
  5. Fairness of Participation
  6. Calculation of Selection Probabilities
  7. Dynamic Model Size Allocation Based on Batch Processing Capabilities
  8. Local training and aggregation
  9. Experiments and results
  10. Conclusion

Figures (4)

  • Figure 1: This figure illustrates the overall framework of CAMA algorithm. In each iteration, the server excludes power domains that have insufficient energy, and clients with too high participation frequency. Then it estimates the number of batches that the clients can handle and dynamically divides the model size for them. Finally, the server samples the clients that meet the requirements.
  • Figure 2: Accuracy for Dirichlet distribution on CIFAR10 dataset. CAMAFL with batch normalization obtains a better average performance than FedZero.
  • Figure 3: Energy consumption over training rounds. CAMAFL gains better average performance, with significantly lower energy consumption.
  • Figure 4: Accuracy for balanced non-IID distribution. CAMAFL obtains a better and more stable accuracy progression compared to FedZero.