Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Carbon Tracking Model for Federated Learning: Impact of Quantization and Sparsification

Luca Barbieri, Stefano Savazzi, Sanaz Kianoush, Monica Nicoli, Luigi Serio

TL;DR

The paper introduces a real-time carbon tracking framework to quantify the energy and carbon footprint of Federated Learning across both centralized (FA) and consensus-based (CFA) architectures. It models per-round emissions and energy costs, incorporating computing, communication, and grid carbon intensity, and evaluates compression strategies—top-$t$ sparsification and probabilistic quantization (qSGD)—on MNIST and CIFAR10 tasks. Key findings show CFA outperforms FA under energy-inefficient communications (low $EE_{COM}$), while FA is preferable when communication is efficient, and that quantization/sparsification can reduce emissions by substantial margins (roughly 42-50%). The results provide practical guidelines for energy-efficient FL design, including optimal compression settings and PS placement relative to grid carbon intensity, with potential for adaptive, heterogeneous deployments in AIoT contexts.

Abstract

Federated Learning (FL) methods adopt efficient communication technologies to distribute machine learning tasks across edge devices, reducing the overhead in terms of data storage and computational complexity compared to centralized solutions. Rather than moving large data volumes from producers (sensors, machines) to energy-hungry data centers, raising environmental concerns due to resource demands, FL provides an alternative solution to mitigate the energy demands of several learning tasks while enabling new Artificial Intelligence of Things (AIoT) applications. This paper proposes a framework for real-time monitoring of the energy and carbon footprint impacts of FL systems. The carbon tracking tool is evaluated for consensus (fully decentralized) and classical FL policies. For the first time, we present a quantitative evaluation of different computationally and communication efficient FL methods from the perspectives of energy consumption and carbon equivalent emissions, suggesting also general guidelines for energy-efficient design. Results indicate that consensus-driven FL implementations should be preferred for limiting carbon emissions when the energy efficiency of the communication is low (i.e., < 25 Kbit/Joule). Besides, quantization and sparsification operations are shown to strike a balance between learning performances and energy consumption, leading to sustainable FL designs.

A Carbon Tracking Model for Federated Learning: Impact of Quantization and Sparsification

TL;DR

The paper introduces a real-time carbon tracking framework to quantify the energy and carbon footprint of Federated Learning across both centralized (FA) and consensus-based (CFA) architectures. It models per-round emissions and energy costs, incorporating computing, communication, and grid carbon intensity, and evaluates compression strategies—top- sparsification and probabilistic quantization (qSGD)—on MNIST and CIFAR10 tasks. Key findings show CFA outperforms FA under energy-inefficient communications (low ), while FA is preferable when communication is efficient, and that quantization/sparsification can reduce emissions by substantial margins (roughly 42-50%). The results provide practical guidelines for energy-efficient FL design, including optimal compression settings and PS placement relative to grid carbon intensity, with potential for adaptive, heterogeneous deployments in AIoT contexts.

Abstract

Federated Learning (FL) methods adopt efficient communication technologies to distribute machine learning tasks across edge devices, reducing the overhead in terms of data storage and computational complexity compared to centralized solutions. Rather than moving large data volumes from producers (sensors, machines) to energy-hungry data centers, raising environmental concerns due to resource demands, FL provides an alternative solution to mitigate the energy demands of several learning tasks while enabling new Artificial Intelligence of Things (AIoT) applications. This paper proposes a framework for real-time monitoring of the energy and carbon footprint impacts of FL systems. The carbon tracking tool is evaluated for consensus (fully decentralized) and classical FL policies. For the first time, we present a quantitative evaluation of different computationally and communication efficient FL methods from the perspectives of energy consumption and carbon equivalent emissions, suggesting also general guidelines for energy-efficient design. Results indicate that consensus-driven FL implementations should be preferred for limiting carbon emissions when the energy efficiency of the communication is low (i.e., < 25 Kbit/Joule). Besides, quantization and sparsification operations are shown to strike a balance between learning performances and energy consumption, leading to sustainable FL designs.
Paper Structure (11 sections, 16 equations, 3 figures, 2 tables)

This paper contains 11 sections, 16 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Carbon tracking framework
  3. Energy tracking for Parameter Server based FL (FA)
  4. Energy tracking for decentralized CFA
  5. Quantization and parameter selection in FL
  6. Parameter Server based FL (FA)
  7. CFA based on CHOCO-SGD framework
  8. Carbon footprint evaluation
  9. Impact of sparsification, energy and carbon efficiencies
  10. CIFAR10 analysis: impact of large model size
  11. Conclusions

Figures (3)

  • Figure 1: Federated Averaging (FA) relying on (a) Parameter Server (PS) for model aggregation and (b) Consensus process based on CHOCO-SGD tool. Sparsification and quantization operations and on-device carbon tracking.
  • Figure 2: Analysis of the validation accuracy achieved by FA and CFA schemes under different levels of quantization and sparsification. Rings group together curves related to the same carbon footprint.
  • Figure 3: Analysis of the validation accuracy under carbon constraints with different communication (energy) efficiencies as well as carbon intensities.