Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Joint Routing and Model Pruning for Decentralized Federated Learning in Bandwidth-Constrained Multi-Hop Wireless Networks

Xiaoyu He, Weicai Li, Tiejun Lv, Xi Yu

Abstract

Decentralized federated learning (D-FL) enables privacy-preserving training without a central server, but multi-hop model exchanges and aggregation are often bottlenecked by communication resource constraints. To address this issue, we propose a joint routing-and-pruning framework that optimizes routing paths and pruning rates to maintain communication latency within prescribed limits. We analyze how the sum of model biases across all clients affects the convergence bound of D-FL and formulate an optimization problem that maximizes the model retention rate to minimize these biases under communication constraints. Further analysis reveals that each client's model retention rate is path-dependent, which reduces the original problem to a routing optimization. Leveraging this insight, we develop a routing algorithm that selects latency-efficient transmission paths, allowing more parameters to be delivered within the time budget and thereby improving D-FL convergence. Simulations demonstrate that, compared with unpruned systems, the proposed framework reduces average transmission latency by 27.8% and improves testing accuracy by approximately 12%. Furthermore, relative to standard benchmark routing algorithms, the proposed routing method improves accuracy by roughly 8%.

Joint Routing and Model Pruning for Decentralized Federated Learning in Bandwidth-Constrained Multi-Hop Wireless Networks

Abstract

Decentralized federated learning (D-FL) enables privacy-preserving training without a central server, but multi-hop model exchanges and aggregation are often bottlenecked by communication resource constraints. To address this issue, we propose a joint routing-and-pruning framework that optimizes routing paths and pruning rates to maintain communication latency within prescribed limits. We analyze how the sum of model biases across all clients affects the convergence bound of D-FL and formulate an optimization problem that maximizes the model retention rate to minimize these biases under communication constraints. Further analysis reveals that each client's model retention rate is path-dependent, which reduces the original problem to a routing optimization. Leveraging this insight, we develop a routing algorithm that selects latency-efficient transmission paths, allowing more parameters to be delivered within the time budget and thereby improving D-FL convergence. Simulations demonstrate that, compared with unpruned systems, the proposed framework reduces average transmission latency by 27.8% and improves testing accuracy by approximately 12%. Furthermore, relative to standard benchmark routing algorithms, the proposed routing method improves accuracy by roughly 8%.
Paper Structure (37 sections, 2 theorems, 32 equations, 12 figures, 3 tables, 2 algorithms)

This paper contains 37 sections, 2 theorems, 32 equations, 12 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Motivation
  3. Challenges
  4. Contributions
  5. Related Work
  6. Model Transmission and Aggregation
  7. Multi-hop Routing Optimization
  8. Model Pruning
  9. Efficient Federated Learning
  10. System model
  11. D-FL model
  12. Model Pruning
  13. Routing and Model Transmission
  14. Collision-Free Transmission Protocol
  15. Routing under Latency Constraints
  16. ...and 22 more sections

Key Result

Lemma 1

Under the common assumptions that the local FL objectives $F_n(\cdot), \forall n$ are $L$-smooth and $\mu$-strongly convex 10965802, the difference between the global model of D-FL at the $\alpha$-th communication round, $\boldsymbol{\bar{\omega}}_{\alpha}$, and the global optimum $\boldsymbol{\omeg where $\tau_{\varrho}$ is a freely adjustable tuning parameter that balances the contributions of t

Figures (12)

  • Figure 1: An illustration of the multi-hop route for transmitting the local model from the source client to all target clients in the $\alpha$-th round.
  • Figure 2: Workflow of the D-FL rounds with local model update, pruning, multi-hop routing, and aggregation.
  • Figure 3: Schematic of structured pruning. This figure shows the structured pruning process applied to input and output channels in a hidden layer. Blue indicates retained channels, while gray and yellow represent pruned input and output channels, respectively. Each small square denotes a model parameter.
  • Figure 4: An illustration of the TDMA-based collision-free protocol for model exchange in a decentralized network.
  • Figure 5: Illustration of the proposed routing Algorithm \ref{['alg:Simplified Optimal Routing']} at different optimization stages and the impact of parameters $\theta$ and $\Psi$ on performance ($N = 20$).
  • ...and 7 more figures

Theorems & Definitions (5)

  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Remark 1: Finite-step Convergence and Local Optimality