Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GLow -- A Novel, Flower-Based Simulated Gossip Learning Strategy

Aitor Belenguer, Jose A. Pascual, Javier Navaridas

TL;DR

The paper tackles the challenge of achieving fully decentralized learning with Gossip Learning by introducing GLow, a simulation-oriented extension built on the Flower Framework. GLow enables researchers to study convergence, scalability, and network effects across customizable topologies without a central aggregation server. Through MNIST and CIFAR-10 experiments with multiple agent counts and connectivity degrees, GLow demonstrates competitive performance relative to centralized and vanilla Federated Learning while highlighting trade-offs arising from topology, data distribution, and special control agents. The work provides a modular, community-friendly tool for advancing decentralized AI on resource-constrained devices and outlines clear avenues for future enhancements such as non-IID handling, parallel head operation, and quantization.

Abstract

Fully decentralized learning algorithms are still in an early stage of development. Creating modular Gossip Learning strategies is not trivial due to convergence challenges and Byzantine faults intrinsic in systems of decentralized nature. Our contribution provides a novel means to simulate custom Gossip Learning systems by leveraging the state-of-the-art Flower Framework. Specifically, we introduce GLow, which will allow researchers to train and assess scalability and convergence of devices, across custom network topologies, before making a physical deployment. The Flower Framework is selected for being a simulation featured library with a very active community on Federated Learning research. However, Flower exclusively includes vanilla Federated Learning strategies and, thus, is not originally designed to perform simulations without a centralized authority. GLow is presented to fill this gap and make simulation of Gossip Learning systems possible. Results achieved by GLow in the MNIST and CIFAR10 datasets, show accuracies over 0.98 and 0.75 respectively. More importantly, GLow performs similarly in terms of accuracy and convergence to its analogous Centralized and Federated approaches in all designed experiments.

GLow -- A Novel, Flower-Based Simulated Gossip Learning Strategy

TL;DR

The paper tackles the challenge of achieving fully decentralized learning with Gossip Learning by introducing GLow, a simulation-oriented extension built on the Flower Framework. GLow enables researchers to study convergence, scalability, and network effects across customizable topologies without a central aggregation server. Through MNIST and CIFAR-10 experiments with multiple agent counts and connectivity degrees, GLow demonstrates competitive performance relative to centralized and vanilla Federated Learning while highlighting trade-offs arising from topology, data distribution, and special control agents. The work provides a modular, community-friendly tool for advancing decentralized AI on resource-constrained devices and outlines clear avenues for future enhancements such as non-IID handling, parallel head operation, and quantization.

Abstract

Fully decentralized learning algorithms are still in an early stage of development. Creating modular Gossip Learning strategies is not trivial due to convergence challenges and Byzantine faults intrinsic in systems of decentralized nature. Our contribution provides a novel means to simulate custom Gossip Learning systems by leveraging the state-of-the-art Flower Framework. Specifically, we introduce GLow, which will allow researchers to train and assess scalability and convergence of devices, across custom network topologies, before making a physical deployment. The Flower Framework is selected for being a simulation featured library with a very active community on Federated Learning research. However, Flower exclusively includes vanilla Federated Learning strategies and, thus, is not originally designed to perform simulations without a centralized authority. GLow is presented to fill this gap and make simulation of Gossip Learning systems possible. Results achieved by GLow in the MNIST and CIFAR10 datasets, show accuracies over 0.98 and 0.75 respectively. More importantly, GLow performs similarly in terms of accuracy and convergence to its analogous Centralized and Federated approaches in all designed experiments.
Paper Structure (17 sections, 7 figures, 2 tables, 1 algorithm)

This paper contains 17 sections, 7 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. State of the art
  3. Self Learning and Centralized Learning
  4. Federated Learning
  5. Gossip Learning
  6. Flower Framework
  7. Related Work
  8. Designed Strategy
  9. Experimental Setup
  10. Datasets and models
  11. Special Agents
  12. Configurations
  13. Results
  14. CNL and FL
  15. GLow
  16. ...and 2 more sections

Figures (7)

  • Figure 1: Topologies in 8+2 agent configuration, from a fully disconnected graph to a fully connected one -- connecting each agent to the next two neighbors respectively. Agents with local data are represented with blue bullets and without local data with white bullets.
  • Figure 2: Loss evolution of CNL (left) and FL (right) systems in the MNIST dataset.
  • Figure 3: Loss evolution of CNL (left) and FL (right) systems in the CIFAR10 dataset.
  • Figure 4: Per-agent accuracy obtained in GLow simulation, MNIST dataset with 8+2 (left) and 16+4 agents (right).
  • Figure 5: Loss evolution of each 8+2 agents during 24 communication rounds (32 local epochs) for topologies 0, 2, 4 and 7 in MNIST.
  • ...and 2 more figures