Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OCD-FL: A Novel Communication-Efficient Peer Selection-based Decentralized Federated Learning

Nizar Masmoudi, Wael Jaafar

TL;DR

A novel scheme, called opportunistic communication-efficient decentralized federated learning, a.k.a., OCD-FL, consisting of a systematic FL peer selection for collaboration, aiming to achieve maximum FL knowledge gain while reducing energy consumption is proposed.

Abstract

The conjunction of edge intelligence and the ever-growing Internet-of-Things (IoT) network heralds a new era of collaborative machine learning, with federated learning (FL) emerging as the most prominent paradigm. With the growing interest in these learning schemes, researchers started addressing some of their most fundamental limitations. Indeed, conventional FL with a central aggregator presents a single point of failure and a network bottleneck. To bypass this issue, decentralized FL where nodes collaborate in a peer-to-peer network has been proposed. Despite the latter's efficiency, communication costs and data heterogeneity remain key challenges in decentralized FL. In this context, we propose a novel scheme, called opportunistic communication-efficient decentralized federated learning, a.k.a., OCD-FL, consisting of a systematic FL peer selection for collaboration, aiming to achieve maximum FL knowledge gain while reducing energy consumption. Experimental results demonstrate the capability of OCD-FL to achieve similar or better performances than the fully collaborative FL, while significantly reducing consumed energy by at least 30% and up to 80%.

OCD-FL: A Novel Communication-Efficient Peer Selection-based Decentralized Federated Learning

TL;DR

A novel scheme, called opportunistic communication-efficient decentralized federated learning, a.k.a., OCD-FL, consisting of a systematic FL peer selection for collaboration, aiming to achieve maximum FL knowledge gain while reducing energy consumption is proposed.

Abstract

The conjunction of edge intelligence and the ever-growing Internet-of-Things (IoT) network heralds a new era of collaborative machine learning, with federated learning (FL) emerging as the most prominent paradigm. With the growing interest in these learning schemes, researchers started addressing some of their most fundamental limitations. Indeed, conventional FL with a central aggregator presents a single point of failure and a network bottleneck. To bypass this issue, decentralized FL where nodes collaborate in a peer-to-peer network has been proposed. Despite the latter's efficiency, communication costs and data heterogeneity remain key challenges in decentralized FL. In this context, we propose a novel scheme, called opportunistic communication-efficient decentralized federated learning, a.k.a., OCD-FL, consisting of a systematic FL peer selection for collaboration, aiming to achieve maximum FL knowledge gain while reducing energy consumption. Experimental results demonstrate the capability of OCD-FL to achieve similar or better performances than the fully collaborative FL, while significantly reducing consumed energy by at least 30% and up to 80%.
Paper Structure (16 sections, 1 theorem, 16 equations, 5 figures, 1 algorithm)

This paper contains 16 sections, 1 theorem, 16 equations, 5 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. System Model
  3. Network Layout
  4. Communication Model
  5. Distributed FL: Background
  6. Dataset Distribution
  7. Local Update
  8. Federated Averaging
  9. Proposed OCD-FL Scheme
  10. Energy Consumption
  11. Knowledge Gain
  12. Problem Formulation
  13. Simulation Experiments and Results
  14. Simulation Setup
  15. Simulation Results
  16. ...and 1 more sections

Key Result

Proposition 1

Let $\mathbf{W^*}$ be the optimal solution of problem eq:local-opt, while $\mathbf{W_1}$ and $\mathbf{W_2}$ are the weight matrices of two different models. For convenience, model efficiency is assumed analogous to its similarity with the optimal solution. Also, the model defined by $\mathbf{W_1}$ o Now, we can deduce the following statements: where $\mathbf{W}^{\rm agg}={\left(\mathbf{W}_1+\math

Figures (5)

  • Figure 1: Effect of regularization on neighbor selection rate.
  • Figure 2: Avg. accuracy and loss (MNIST, different schemes).
  • Figure 3: Avg. accuracy and loss (CIFAR-10, different schemes).
  • Figure 4: Knowledge gain (different schemes and scenarios).
  • Figure 5: Consumed communication energy (different schemes and scenarios).

Theorems & Definitions (1)

  • Proposition 1