Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Collaborations through Weight-Driven Coalition Dynamics in Federated Learning Systems

Mohammed El Hanjri, Hamza Reguieg, Adil Attiaoui, Amine Abouaomar, Abdellatif Kobbane, Mohamed El Kamili

TL;DR

This work tackles the challenge of heterogeneity in IoT-enabled federated learning by introducing a weight-space coalitional framework. Devices are grouped into coalitions using the Euclidean distance between local model weights, and each coalition's barycenter acts as an aggregation anchor, enabling a centralized global update with reduced communication. Empirical results on MNIST show that coalition-based FL achieves more stable convergence and robustness under non-IID data, while maintaining performance comparable to or better than FedAvg in homogeneous settings. The approach offers a practical, scalable path to efficient decentralized learning in resource-constrained IoT environments, leveraging weight-space similarity to guide collaboration.

Abstract

In the era of the Internet of Things (IoT), decentralized paradigms for machine learning are gaining prominence. In this paper, we introduce a federated learning model that capitalizes on the Euclidean distance between device model weights to assess their similarity and disparity. This is foundational for our system, directing the formation of coalitions among devices based on the closeness of their model weights. Furthermore, the concept of a barycenter, representing the average of model weights, helps in the aggregation of updates from multiple devices. We evaluate our approach using homogeneous and heterogeneous data distribution, comparing it against traditional federated learning averaging algorithm. Numerical results demonstrate its potential in offering structured, outperformed and communication-efficient model for IoT-based machine learning.

Efficient Collaborations through Weight-Driven Coalition Dynamics in Federated Learning Systems

TL;DR

This work tackles the challenge of heterogeneity in IoT-enabled federated learning by introducing a weight-space coalitional framework. Devices are grouped into coalitions using the Euclidean distance between local model weights, and each coalition's barycenter acts as an aggregation anchor, enabling a centralized global update with reduced communication. Empirical results on MNIST show that coalition-based FL achieves more stable convergence and robustness under non-IID data, while maintaining performance comparable to or better than FedAvg in homogeneous settings. The approach offers a practical, scalable path to efficient decentralized learning in resource-constrained IoT environments, leveraging weight-space similarity to guide collaboration.

Abstract

In the era of the Internet of Things (IoT), decentralized paradigms for machine learning are gaining prominence. In this paper, we introduce a federated learning model that capitalizes on the Euclidean distance between device model weights to assess their similarity and disparity. This is foundational for our system, directing the formation of coalitions among devices based on the closeness of their model weights. Furthermore, the concept of a barycenter, representing the average of model weights, helps in the aggregation of updates from multiple devices. We evaluate our approach using homogeneous and heterogeneous data distribution, comparing it against traditional federated learning averaging algorithm. Numerical results demonstrate its potential in offering structured, outperformed and communication-efficient model for IoT-based machine learning.
Paper Structure (14 sections, 5 equations, 4 figures, 1 algorithm)

This paper contains 14 sections, 5 equations, 4 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. System model
  4. Euclidean distance between weights
  5. Barycenter of a subset of users
  6. Model Description
  7. Simulation and Numerical Results
  8. Simulation Setup
  9. Dataset
  10. Algorithm Implementation
  11. Model Architecture
  12. Parameters and Configuration
  13. Discussion
  14. Conclusion

Figures (4)

  • Figure 1: An illustrative overview of the different stages of coalition formation in one global training round.
  • Figure 2: Accuracy of FedAvg and the proposed approach in case of homogeneous data distribution
  • Figure 3: Accuracy of FedAvg and the proposed approach in case of heterogeneous data distribution
  • Figure 4: Accuracy of FedAvg and the proposed approach in case of highly heterogeneous data distribution