Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tram-FL: Routing-based Model Training for Decentralized Federated Learning

Kota Maejima, Takayuki Nishio, Asato Yamazaki, Yuko Hara-Azumi

TL;DR

This work proposes Tram-FL, which progressively refines a global model by transferring it sequentially amongst nodes, and introduces a dynamic model routing algorithm for optimal route selection, aimed at enhancing model precision with minimal forwarding.

Abstract

In decentralized federated learning (DFL), substantial traffic from frequent inter-node communication and non-independent and identically distributed (non-IID) data challenges high-accuracy model acquisition. We propose Tram-FL, a novel DFL method, which progressively refines a global model by transferring it sequentially amongst nodes, rather than by exchanging and aggregating local models. We also introduce a dynamic model routing algorithm for optimal route selection, aimed at enhancing model precision with minimal forwarding. Our experiments using MNIST, CIFAR-10, and IMDb datasets demonstrate that Tram-FL with the proposed routing delivers high model accuracy under non-IID conditions, outperforming baselines while reducing communication costs.

Tram-FL: Routing-based Model Training for Decentralized Federated Learning

TL;DR

This work proposes Tram-FL, which progressively refines a global model by transferring it sequentially amongst nodes, and introduces a dynamic model routing algorithm for optimal route selection, aimed at enhancing model precision with minimal forwarding.

Abstract

In decentralized federated learning (DFL), substantial traffic from frequent inter-node communication and non-independent and identically distributed (non-IID) data challenges high-accuracy model acquisition. We propose Tram-FL, a novel DFL method, which progressively refines a global model by transferring it sequentially amongst nodes, rather than by exchanging and aggregating local models. We also introduce a dynamic model routing algorithm for optimal route selection, aimed at enhancing model precision with minimal forwarding. Our experiments using MNIST, CIFAR-10, and IMDb datasets demonstrate that Tram-FL with the proposed routing delivers high model accuracy under non-IID conditions, outperforming baselines while reducing communication costs.
Paper Structure (8 sections, 3 equations, 4 figures, 3 tables, 1 algorithm)

This paper contains 8 sections, 3 equations, 4 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. PROPOSED METHOD
  3. System model
  4. Tram-FL Algorithm
  5. Model Routing Algorithm
  6. EXPERIMENTAL SETUP
  7. Experimental Result
  8. Conclusion

Figures (4)

  • Figure 1: Illustration of the sequence in the proposed Tram-FL model updating procedure. Step 1 involves the node updating the model parameter using a minibatch, followed by the transmission of the model to the next node in Step 2.
  • Figure 2: Test accuracy as a function of the total number of model transmissions, utilizing non-IID data sets. Evaluation includes (a) MNIST for handwritten digit classification, (b) CIFAR-10 for object recognition, and (c) IMDb for sentiment analysis.
  • Figure 3: Comparison of the number of model transmissions required to reach a 78% accuracy level, illustrated across different model routing methods.
  • Figure : Tram-FL