Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Context-Aware Orchestration of Energy-Efficient Gossip Learning Schemes

Mina Aghaei Dinani, Adrian Holzer, Hung Nguyen, Marco Ajmone Marsan, Gianluca Rizzo

TL;DR

The paper addresses the high energy cost of fully distributed Gossip Learning in dynamic networks by introducing Optimized Gossip Learning (OGL), which uses a DNN-based orchestrator to adapt per-node training epochs and neighbor model merges in real time. The approach formalizes an energy-aware optimization problem and deploys an offline-trained $M_{tune}$ to guide local computations and model exchanges, achieving target accuracy with reduced energy consumption. Evaluations on MNIST and CIFAR-10 across time-varying random graphs and a dynamic urban mobility scenario show that OGL attains centralized-level accuracy while significantly lowering energy costs, demonstrating robustness to network dynamics and node churn. The work advances scalable, energy-conscious distributed learning for battery-constrained edge and IoT deployments, with potential for fully distributed self-optimization in future iterations.

Abstract

Fully distributed learning schemes such as Gossip Learning (GL) are gaining momentum due to their scalability and effectiveness even in dynamic settings. However, they often imply a high utilization of communication and computing resources, whose energy footprint may jeopardize the learning process, particularly on battery-operated IoT devices. To address this issue, we present Optimized Gossip Learning (OGL)}, a distributed training approach based on the combination of GL with adaptive optimization of the learning process, which allows for achieving a target accuracy while minimizing the energy consumption of the learning process. We propose a data-driven approach to OGL management that relies on optimizing in real-time for each node the number of training epochs and the choice of which model to exchange with neighbors based on patterns of node contacts, models' quality, and available resources at each node. Our approach employs a DNN model for dynamic tuning of the aforementioned parameters, trained by an infrastructure-based orchestrator function. We performed our assessments on two different datasets, leveraging time-varying random graphs and a measurement-based dynamic urban scenario. Results suggest that our approach is highly efficient and effective in a broad spectrum of network scenarios.

Context-Aware Orchestration of Energy-Efficient Gossip Learning Schemes

TL;DR

The paper addresses the high energy cost of fully distributed Gossip Learning in dynamic networks by introducing Optimized Gossip Learning (OGL), which uses a DNN-based orchestrator to adapt per-node training epochs and neighbor model merges in real time. The approach formalizes an energy-aware optimization problem and deploys an offline-trained to guide local computations and model exchanges, achieving target accuracy with reduced energy consumption. Evaluations on MNIST and CIFAR-10 across time-varying random graphs and a dynamic urban mobility scenario show that OGL attains centralized-level accuracy while significantly lowering energy costs, demonstrating robustness to network dynamics and node churn. The work advances scalable, energy-conscious distributed learning for battery-constrained edge and IoT deployments, with potential for fully distributed self-optimization in future iterations.

Abstract

Fully distributed learning schemes such as Gossip Learning (GL) are gaining momentum due to their scalability and effectiveness even in dynamic settings. However, they often imply a high utilization of communication and computing resources, whose energy footprint may jeopardize the learning process, particularly on battery-operated IoT devices. To address this issue, we present Optimized Gossip Learning (OGL)}, a distributed training approach based on the combination of GL with adaptive optimization of the learning process, which allows for achieving a target accuracy while minimizing the energy consumption of the learning process. We propose a data-driven approach to OGL management that relies on optimizing in real-time for each node the number of training epochs and the choice of which model to exchange with neighbors based on patterns of node contacts, models' quality, and available resources at each node. Our approach employs a DNN model for dynamic tuning of the aforementioned parameters, trained by an infrastructure-based orchestrator function. We performed our assessments on two different datasets, leveraging time-varying random graphs and a measurement-based dynamic urban scenario. Results suggest that our approach is highly efficient and effective in a broad spectrum of network scenarios.
Paper Structure (8 sections, 5 equations, 8 figures, 2 tables)

This paper contains 8 sections, 5 equations, 8 figures, 2 tables.

Table of Contents

  1. Introduction
  2. System model
  3. The OGL Approach to Energy-Efficient Gossip Learning
  4. A Gossip-Based Collaborative Training Algorithm
  5. Formulation of the energy optimization problem
  6. OGL architecture, components and functions
  7. Numerical assessment
  8. Conclusions

Figures (8)

  • Figure 1: Comparative analysis of cost function on the MNIST dataset with $|V|=6$ and various values for $p$.
  • Figure 2: Comparative analysis of cost function and mean accuracy at convergence on the CIFAR-10 dataset, with $|V|=6$ and $p=1$.
  • Figure 3: Mean accuracy versus round of our OGL algorithm compared to baselines using the CIFAR-10 dataset, with $|V|=6$ and $p = 1$. Curves are associated with a $95\%$ confidence interval.
  • Figure 4: Mean accuracy versus round for OGL, random GL, and DP algorithm using the MNIST dataset, for different values of number of nodes in the system and edge probability. Plots are with $95\%$ confidence interval.
  • Figure 5: Mean accuracy versus round for the OGL, Random GL, and DP algorithms, for the CIFAR-10 dataset, for different values of number of nodes in the system and edge probability. Plots are with $95\%$ confidence interval.
  • ...and 3 more figures