Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TT-Prune: Joint Model Pruning and Resource Allocation for Communication-efficient Time-triggered Federated Learning

Xinlu Zhang, Yansha Deng, Toktam Mahmoodi

TL;DR

This work tackles the communication bottlenecks in wireless time-triggered federated learning by introducing TT-Prune, a framework that jointly optimizes adaptive model pruning and wireless bandwidth across multiple tiers. It derives a convergence upper bound on the gradient norm under pruning and formulates a latency-constrained optimization, solved via decoupled subproblems using Karush-Kuhn-Tucker conditions to yield closed-form pruning ratios and bandwidth allocations. Simulations show TT-Prune can reduce total training latency by about 40% without sacrificing accuracy, demonstrating improved scalability and efficiency for asynchronous/multi-tier FL in wireless environments. The approach offers practical benefits for edge deployments and V2X scenarios, where dynamic channel conditions and heterogeneous devices drive the need for latency-aware, pruning-enabled TT-Fed.

Abstract

Federated learning (FL) offers new opportunities in machine learning, particularly in addressing data privacy concerns. In contrast to conventional event-based federated learning, time-triggered federated learning (TT-Fed), as a general form of both asynchronous and synchronous FL, clusters users into different tiers based on fixed time intervals. However, the FL network consists of a growing number of user devices with limited wireless bandwidth, consequently magnifying issues such as stragglers and communication overhead. In this paper, we introduce adaptive model pruning to wireless TT-Fed systems and study the problem of jointly optimizing the pruning ratio and bandwidth allocation to minimize the training loss while ensuring minimal learning latency. To answer this question, we perform convergence analysis on the gradient l_2 norm of the TT-Fed model based on model pruning. Based on the obtained convergence upper bound, a joint optimization problem of pruning ratio and wireless bandwidth is formulated to minimize the model training loss under a given delay threshold. Then, we derive closed-form solutions for wireless bandwidth and pruning ratio using Karush-Kuhn-Tucker(KKT) conditions. The simulation results show that model pruning could reduce the communication cost by 40% while maintaining the model performance at the same level.

TT-Prune: Joint Model Pruning and Resource Allocation for Communication-efficient Time-triggered Federated Learning

TL;DR

This work tackles the communication bottlenecks in wireless time-triggered federated learning by introducing TT-Prune, a framework that jointly optimizes adaptive model pruning and wireless bandwidth across multiple tiers. It derives a convergence upper bound on the gradient norm under pruning and formulates a latency-constrained optimization, solved via decoupled subproblems using Karush-Kuhn-Tucker conditions to yield closed-form pruning ratios and bandwidth allocations. Simulations show TT-Prune can reduce total training latency by about 40% without sacrificing accuracy, demonstrating improved scalability and efficiency for asynchronous/multi-tier FL in wireless environments. The approach offers practical benefits for edge deployments and V2X scenarios, where dynamic channel conditions and heterogeneous devices drive the need for latency-aware, pruning-enabled TT-Fed.

Abstract

Federated learning (FL) offers new opportunities in machine learning, particularly in addressing data privacy concerns. In contrast to conventional event-based federated learning, time-triggered federated learning (TT-Fed), as a general form of both asynchronous and synchronous FL, clusters users into different tiers based on fixed time intervals. However, the FL network consists of a growing number of user devices with limited wireless bandwidth, consequently magnifying issues such as stragglers and communication overhead. In this paper, we introduce adaptive model pruning to wireless TT-Fed systems and study the problem of jointly optimizing the pruning ratio and bandwidth allocation to minimize the training loss while ensuring minimal learning latency. To answer this question, we perform convergence analysis on the gradient l_2 norm of the TT-Fed model based on model pruning. Based on the obtained convergence upper bound, a joint optimization problem of pruning ratio and wireless bandwidth is formulated to minimize the model training loss under a given delay threshold. Then, we derive closed-form solutions for wireless bandwidth and pruning ratio using Karush-Kuhn-Tucker(KKT) conditions. The simulation results show that model pruning could reduce the communication cost by 40% while maintaining the model performance at the same level.

Paper Structure

This paper contains 30 sections, 51 equations, 6 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. System Model
  3. TT-Fed Framework
  4. Global Model Broadcast
  5. Local Model Learning
  6. Intra-tier Aggregation
  7. Global Aggregation
  8. Federated Learning With Model Pruning
  9. Latency In Learning Process
  10. Local Model Update With Pruning
  11. Local Model Uplink Transmission
  12. Intra-tier Aggregation
  13. Global Model Aggregation and Broadcast
  14. Problem Formulation
  15. Convergence Analysis
  16. ...and 15 more sections

Figures (6)

  • Figure 1: The work-flow of Pruned Time-Triggered FL under the given aggregation duration $\Delta T$.
  • Figure 2: Test accuracy for four different schemes on two-Tier TT-Fed
  • Figure 3: Test accuracy of TT-Fed under different time constraints on CIFAR
  • Figure 4: Model performance for four different schemes on two-Tier TT-Fed
  • Figure 5: Performance required for TT-Prune and other schemes to achieve 80% accuracy in Non-IID FMNIST dataset
  • ...and 1 more figures