Federated Split Learning with Model Pruning and Gradient Quantization in Wireless Networks

TL;DR

The paper tackles the bottleneck of training large models via federated split learning on resource-limited edge devices. It proposes a lightweight FedSL framework that combines client-side model pruning, gradient quantization, and dropout of split-layer activations to cut computation and communication costs, with periodic client-side model aggregation every $I$ rounds. A convergence analysis derives an upper bound on the average gradient norm that explicitly depends on aggregation frequency, split-layer position, pruning rate, and quantization precision, and is validated by simulations on CIFAR-10 with VGG-19 showing faster training and competitive accuracy. The results indicate that moderate pruning and 8-bit quantization provide tangible speedups and robustness, especially with shallower split layers, making FedSL more practical for wireless edge networks.

Abstract

As a paradigm of distributed machine learning, federated learning typically requires all edge devices to train a complete model locally. However, with the increasing scale of artificial intelligence models, the limited resources on edge devices often become a bottleneck for efficient fine-tuning. To address this challenge, federated split learning (FedSL) implements collaborative training across the edge devices and the server through model splitting. In this paper, we propose a lightweight FedSL scheme, that further alleviates the training burden on resource-constrained edge devices by pruning the client-side model dynamicly and using quantized gradient updates to reduce computation overhead. Additionally, we apply random dropout to the activation values at the split layer to reduce communication overhead. We conduct theoretical analysis to quantify the convergence performance of the proposed scheme. Finally, simulation results verify the effectiveness and advantages of the proposed lightweight FedSL in wireless network environments.

Figures (4)

• Figure 1: An illustration of the proposed FedSL with client-side model pruning and gradient quantization.
• Figure 2: Impact of pruning and quantization on performance without periodic aggregation and dropout.
• Figure 3: Impact of aggregation frequency $I$, split layer selection $L_c$, number of clients $K$ and dropout rate $p_i$ on performance with pruning rate $\rho_f = 0.35$ and quantized bits $q=8$.
• Figure 4: Impact of dropout rate on latency with varying split layer selection under pruning rate $\rho_f=0.35$, quantized bits $q=8$, aggregation frequency $I=5$ and number of clients $K=5$.

Theorems & Definitions (3)

• proof
• proof
• proof