Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Energy-Efficient Federated Learning for AIoT using Clustering Methods

Roberto Pereira, Fernanda Famá, Charalampos Kalalas, Paolo Dini

TL;DR

The paper tackles the energy efficiency of federated learning in AIoT by addressing the high energy cost of local training and data heterogeneity. It proposes two clustering-informed, one-time preprocessing methods, SimClust and RepClust, to guide server-side client selection and reduce redundant computations while preserving convergence. Through experiments on F-MNIST, CIFAR-10, and CIFAR-100, the authors show that these methods achieve strong convergence with lower total energy than baselines like Rand and GP-based FedCor, and they analyze privacy implications via differential privacy noise on label distributions. The approach offers a practical pathway for sustainable distributed learning in resource-constrained edge environments and suggests future extensions with over-the-air computing and decentralized clustering.

Abstract

While substantial research has been devoted to optimizing model performance, convergence rates, and communication efficiency, the energy implications of federated learning (FL) within Artificial Intelligence of Things (AIoT) scenarios are often overlooked in the existing literature. This study examines the energy consumed during the FL process, focusing on three main energy-intensive processes: pre-processing, communication, and local learning, all contributing to the overall energy footprint. We rely on the observation that device/client selection is crucial for speeding up the convergence of model training in a distributed AIoT setting and propose two clustering-informed methods. These clustering solutions are designed to group AIoT devices with similar label distributions, resulting in clusters composed of nearly heterogeneous devices. Hence, our methods alleviate the heterogeneity often encountered in real-world distributed learning applications. Throughout extensive numerical experimentation, we demonstrate that our clustering strategies typically achieve high convergence rates while maintaining low energy consumption when compared to other recent approaches available in the literature.

Energy-Efficient Federated Learning for AIoT using Clustering Methods

TL;DR

The paper tackles the energy efficiency of federated learning in AIoT by addressing the high energy cost of local training and data heterogeneity. It proposes two clustering-informed, one-time preprocessing methods, SimClust and RepClust, to guide server-side client selection and reduce redundant computations while preserving convergence. Through experiments on F-MNIST, CIFAR-10, and CIFAR-100, the authors show that these methods achieve strong convergence with lower total energy than baselines like Rand and GP-based FedCor, and they analyze privacy implications via differential privacy noise on label distributions. The approach offers a practical pathway for sustainable distributed learning in resource-constrained edge environments and suggests future extensions with over-the-air computing and decentralized clustering.

Abstract

While substantial research has been devoted to optimizing model performance, convergence rates, and communication efficiency, the energy implications of federated learning (FL) within Artificial Intelligence of Things (AIoT) scenarios are often overlooked in the existing literature. This study examines the energy consumed during the FL process, focusing on three main energy-intensive processes: pre-processing, communication, and local learning, all contributing to the overall energy footprint. We rely on the observation that device/client selection is crucial for speeding up the convergence of model training in a distributed AIoT setting and propose two clustering-informed methods. These clustering solutions are designed to group AIoT devices with similar label distributions, resulting in clusters composed of nearly heterogeneous devices. Hence, our methods alleviate the heterogeneity often encountered in real-world distributed learning applications. Throughout extensive numerical experimentation, we demonstrate that our clustering strategies typically achieve high convergence rates while maintaining low energy consumption when compared to other recent approaches available in the literature.
Paper Structure (23 sections, 11 equations, 8 figures, 2 tables, 1 algorithm)

This paper contains 23 sections, 11 equations, 8 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Contributions
  4. Preliminaries
  5. Active Client Selection via Priority Ranking
  6. Clustering-Informed Client Selection Mechanisms
  7. Clustering Similar Clients Together
  8. Maximizing Group Diversity
  9. Scalability of Clustering Solutions
  10. Privacy Concerns
  11. Energy Consumption of Federated Learning Process
  12. Hardware Energy Consumption: CPU, GPU and Memory
  13. Over the Air communication
  14. Numerical Evaluation
  15. Experiment Setup
  16. ...and 8 more sections

Figures (8)

  • Figure 1: Percentage of the total energy consumed during training (solid) and communication (striped) stages of a heterogeneous FL process with $100$ clients learning on CIFAR-10 dataset. Random selection of $10$ clients was performed over $100$ communication rounds. The number of learnable parameters of each model is reported inside the parenthesis and alongside the model name on the x-axis.
  • Figure 2: Comparison between traditional device random sampling (first row) and the proposed device sampling (second row). The circular disks next to each device represent their data and label distribution (i.e., each color denotes a different label). In random sampling, the numbers next to the data represent the ranking of the device. In the second row, clients with the same colors exhibit similar label distributions.
  • Figure 3: Illustration of different data partition schemes for CIFAR-10 datasets for fixed $\alpha = 1.0$ and varying $\rho$. The higher the $\rho$, the higher the data heterogeneity.
  • Figure 4: Energy profiles of client selection mechanisms (represented by different colors) divided into learning (solid color at the bottom), pre-processing (diagonal pattern in the middle), and communication (cross pattern at the top). Energy values are reported as relative to the total energy consumed in the FL process with a random selection of $10$ out of $100$ clients for CIFAR-10 (a)-(c) and F-MNIST (d)-(f). Final accuracies are reported inline above each scenario.
  • Figure 5: Accuracy plots for (a) CIFAR and (b) F-MNIST for different heterogeneity scenarios, $\rho=1,2,5$. Variance is depicted as a shaded area.
  • ...and 3 more figures