Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Supplementary File: Coded Cooperative Networks for Semi-Decentralized Federated Learning

Shudi Weng, Ming Xiao, Chao Ren, Mikael Skoglund

TL;DR

The paper tackles FL performance degradation due to communication stragglers in wireless settings by introducing a deterministic coded cooperative network that enables semi-decentralized training without requiring global network knowledge. It leverages diversity through MDS-based network coding (DNC) to ensure that PS can recover a meaningful subset of client updates despite intermittent links, and it provides outage and convergence analyses to support robustness. The approach is validated on MNIST, showing the proposed scheme can match perfect-link FL in both i.i.d. and non-i.i.d. data scenarios while outperforming variants that rely on prior network information or lack coding diversity. This work offers a scalable, practically deployable mechanism to mitigate communication bottlenecks in distributed FL over wireless networks and suggests applicability to large-scale models beyond MNIST, including potential LLM training scenarios.

Abstract

To enhance straggler resilience in federated learning (FL) systems, a semi-decentralized approach has been recently proposed, enabling collaboration between clients. Unlike the existing semi-decentralized schemes, which adaptively adjust the collaboration weight according to the network topology, this letter proposes a deterministic coded network that leverages wireless diversity for semi-decentralized FL without requiring prior information about the entire network. Furthermore, the theoretical analyses of the outage and the convergence rate of the proposed scheme are provided. Finally, the superiority of our proposed method over benchmark methods is demonstrated through comprehensive simulations.

Supplementary File: Coded Cooperative Networks for Semi-Decentralized Federated Learning

TL;DR

The paper tackles FL performance degradation due to communication stragglers in wireless settings by introducing a deterministic coded cooperative network that enables semi-decentralized training without requiring global network knowledge. It leverages diversity through MDS-based network coding (DNC) to ensure that PS can recover a meaningful subset of client updates despite intermittent links, and it provides outage and convergence analyses to support robustness. The approach is validated on MNIST, showing the proposed scheme can match perfect-link FL in both i.i.d. and non-i.i.d. data scenarios while outperforming variants that rely on prior network information or lack coding diversity. This work offers a scalable, practically deployable mechanism to mitigate communication bottlenecks in distributed FL over wireless networks and suggests applicability to large-scale models beyond MNIST, including potential LLM training scenarios.

Abstract

To enhance straggler resilience in federated learning (FL) systems, a semi-decentralized approach has been recently proposed, enabling collaboration between clients. Unlike the existing semi-decentralized schemes, which adaptively adjust the collaboration weight according to the network topology, this letter proposes a deterministic coded network that leverages wireless diversity for semi-decentralized FL without requiring prior information about the entire network. Furthermore, the theoretical analyses of the outage and the convergence rate of the proposed scheme are provided. Finally, the superiority of our proposed method over benchmark methods is demonstrated through comprehensive simulations.

Paper Structure

This paper contains 25 sections, 6 theorems, 39 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Related works
  3. Our contributions
  4. System Model for Semi-decentralized FL over Wireless Network
  5. Local Training at Clients
  6. Transmission over Wireless Network
  7. Network Model
  8. Quantized Transmission and Outage
  9. Aggregation at PS
  10. The Proposed Method: Coded Cooperative Network for Semi-decentralized FL
  11. System Description
  12. Broadcasting
  13. Local training
  14. Transmission
  15. PS Decoding and Aggregation
  16. ...and 10 more sections

Key Result

Lemma 1

Given that PS is equally likely to see each local model update from any client with $1-q$, and the aggregation of these updates to recover the global model is statistically unbiased in terms of the expected value, that is, Besides, it can be proved that where $\Bar{\alpha}_m=\frac{1}{M\Bar{K}}$ with $\frac{1}{\Bar{K}}=\sum_{l=1}^M\frac{\frac{1}{l}\mathbb{C}_M^l(1-q)^lq^{M-l}}{1-q^M}\leq \frac{2}

Figures (3)

  • Figure 1: Illustration of the proposed scheme within the semi-decentralized FL system over the intermittent links in $M$ slots with two communication stages.
  • Figure 2: Test accuracy comparison of four methods with $R=0.6$ under different SNRs in terms of communication round in the i.i.d. setting.
  • Figure 3: Test accuracy comparison of four methods under different levels of data imbalances in terms of communication round with $\mathrm{SNR}=3$, $R=0.6$.

Theorems & Definitions (17)

  • Definition 1
  • Lemma 1
  • proof
  • Lemma 2
  • Theorem 1
  • proof
  • proof : Proof 1
  • proof : Proof 2
  • proof : Proof 3
  • Lemma 3
  • ...and 7 more