EcoLearn: Optimizing the Carbon Footprint of Federated Learning

TL;DR

EcoLearn tackles the carbon footprint of federated learning by coordinating when, how many, and which clients participate, guided by carbon intensity and data utility. It introduces three modules (CScale, CProv, CSelect) and an integrated policy that seeks a favorable carbon-accuracy-time Pareto frontier. The approach is evaluated on both synthetic and real-world datasets, showing up to 10.8x carbon reduction with negligible degradation in accuracy and training time. The work highlights the importance of accounting for spatial and temporal carbon variability in FL and offers a practical blueprint for greener distributed learning.

Abstract

Federated Learning (FL) distributes machine learning (ML) training across edge devices to reduce data transfer overhead and protect data privacy. Since FL model training may span hundreds of devices and is thus resource- and energy-intensive, it has a significant carbon footprint. Importantly, since energy's carbon-intensity differs substantially (by up to 60$\times$) across locations, training on the same device using the same amount of energy, but at different locations, can incur widely different carbon emissions. While prior work has focused on improving FL's resource- and energy-efficiency by optimizing time-to-accuracy, it implicitly assumes all energy has the same carbon intensity and thus does not optimize carbon efficiency, i.e., work done per unit of carbon emitted. To address the problem, we design EcoLearn, which minimizes FL's carbon footprint without significantly affecting model accuracy or training time. EcoLearn achieves a favorable tradeoff by integrating carbon awareness into multiple aspects of FL training, including i) selecting clients with high data utility and low carbon, ii) provisioning more clients during the initial training rounds, and iii) mitigating stragglers by dynamically adjusting client over-provisioning based on carbon. We implement EcoLearn and its carbon-aware FL training policies in the Flower framework and show that it reduces the carbon footprint of training (by up to $10.8$$\times$) while maintaining model accuracy and training time (within $\sim$$1$\%) compared to state-of-the-art approaches.

Figures (19)

• Figure 1: Grey box highlights the selection of more clients in the initial rounds. Purple box shows that we use our basic utility and carbon-aware variant in the EcoLearn.
• Figure 2: Histogram of average carbon-intensity across 100 locations worldwide. Carbon-intensity ranges from near 0g$\cdot$CO$_2$g/kWh to nearly 1000g$\cdot$CO$_2$g/kWh.
• Figure 3: Impact of data heterogeneity across clients on the statistical utility of a given client toward the global model.
• Figure 4: Impact of the number of training rounds on the statistical utility of a client at a fixed non-iid level.
• Figure 5: Impact of how many clients are selected across rounds on the accuracy of the globally trained model.