FedGreen: Carbon-aware Federated Learning with Model Size Adaptation

TL;DR

FedGreen addresses the carbon footprint of federated learning by introducing carbon-aware model size adaptation using ordered dropout. It clusters clients by carbon intensity and assigns cluster-specific width scaling factors $p_m$, formulating $C_{FL}(oldsymbol{p}) = \sum_{m=1}^M p_m^2 A_m$ and linking training rounds $R$ to the statistics of $\boldsymbol{p}$ via $R \propto \frac{\sigma(\boldsymbol{p})^{\beta}}{\mu(\boldsymbol{p})^{\lambda}}$. The method enables heterogeneous aggregation and submodel dissemination across rounds, achieving substantial carbon-emission reductions with little loss in accuracy, as demonstrated on EMNIST with real and simulated carbon-intensity profiles. The work provides theoretical insights into the trade-offs between carbon efficiency and convergence and offers practical guidelines for deploying greener FL in heterogeneous, geo-distributed environments.

Abstract

Federated learning (FL) provides a promising collaborative framework to build a model from distributed clients, and this work investigates the carbon emission of the FL process. Cloud and edge servers hosting FL clients may exhibit diverse carbon footprints influenced by their geographical locations with varying power sources, offering opportunities to reduce carbon emissions by training local models with adaptive computations and communications. In this paper, we propose FedGreen, a carbon-aware FL approach to efficiently train models by adopting adaptive model sizes shared with clients based on their carbon profiles and locations using ordered dropout as a model compression technique. We theoretically analyze the trade-offs between the produced carbon emissions and the convergence accuracy, considering the carbon intensity discrepancy across countries to choose the parameters optimally. Empirical studies show that FedGreen can substantially reduce the carbon footprints of FL compared to the state-of-the-art while maintaining competitive model accuracy.

Figures (3)

• Figure 1: The proposed FedGreen model size adaptation method. $p_i$ is the remaining rate of neurons. For the clients on hosts with higher carbon footprints, smaller models are sent. For those with lower carbon footprints, larger models are sent.
• Figure 2: This batch of figures represents a Sensitivity Analysis for Clustering, specifically focusing on varying hyperparameters that exhibit non-identically distributed (non-IID) characteristics and the number of local epochs.
• Figure 3: This batch of figures represents a sensitivity analysis for carbon intensity profile using two different approaches(Simulated and Real) for the carbon profile of the clients in two and three clusters scenarios.