Efficient Parallel Split Learning over Resource-constrained Wireless Edge Networks

TL;DR

This work tackles the latency bottleneck of privacy-preserving distributed learning on wireless edge networks by introducing Efficient Parallel Split Learning (EPSL), which combines parallel client-side training with last-layer gradient aggregation to reduce server-side computation and communication. A tunable aggregation ratio $\phi \in [0,1]$ enables a trade-off between accuracy and latency, with PSL recovered when $\phi=0$. The authors formulate a joint resource-management and cut-layer design problem—encompassing subchannel allocation and power control— and solve it with a practical BCD-based algorithm that decomposes into tractable subproblems. Experiments on MNIST and HAM10000 with ResNet-18 demonstrate that EPSL can achieve comparable accuracy to baseline PSL/SFL/SL methods while significantly lowering per-round training time, and that the tailored resource-management strategy yields additional latency reductions. Overall, EPSL offers a scalable, privacy-preserving learning paradigm well-suited to resource-constrained edge deployments.

Abstract

The increasingly deeper neural networks hinder the democratization of privacy-enhancing distributed learning, such as federated learning (FL), to resource-constrained devices. To overcome this challenge, in this paper, we advocate the integration of edge computing paradigm and parallel split learning (PSL), allowing multiple client devices to offload substantial training workloads to an edge server via layer-wise model split. By observing that existing PSL schemes incur excessive training latency and large volume of data transmissions, we propose an innovative PSL framework, namely, efficient parallel split learning (EPSL), to accelerate model training. To be specific, EPSL parallelizes client-side model training and reduces the dimension of local gradients for back propagation (BP) via last-layer gradient aggregation, leading to a significant reduction in server-side training and communication latency. Moreover, by considering the heterogeneous channel conditions and computing capabilities at client devices, we jointly optimize subchannel allocation, power control, and cut layer selection to minimize the per-round latency. Simulation results show that the proposed EPSL framework significantly decreases the training latency needed to achieve a target accuracy compared with the state-of-the-art benchmarks, and the tailored resource management and layer split strategy can considerably reduce latency than the counterpart without optimization.

Figures (13)

• Figure 1: An illustration of vanilla SL, PSL, SFL and EPSL frameworks, where $\phi$ denotes the ratio of last-layer gradient aggregation. When $\phi = 0$, EPSL is reduced to PSL.
• Figure 2: The illustration of EPSL over wireless networks.
• Figure 3: An example of the last-layer gradient aggregation in EPSL with $\phi=0.5$ (e.g., half of data going through last-layer aggregation), where two edge devices with two data samples participate in model training.
• Figure 4: The test accuracy (a) and per-round latency (b) of EPSL, PSL, SFL, and vanilla SL on HAM 10000 under IID settings using ResNet-18 with $C=5$ edge devices. As shown in the figures, EPSL achieves similar accuracy compared with other benchmarks with the same number of rounds, yet with much shorter per-round latency.
• Figure 5: An illustration of EPSL training procedure for one training round.