ESFL: Efficient Split Federated Learning over Resource-Constrained Heterogeneous Wireless Devices

TL;DR

ESFL tackles the straggler problem in resource-heterogeneous edge environments by jointly optimizing how much of the model resides on the user side (cut layer) and how server-side computing resources are allocated. It casts the training process as a MINLP and solves it via alternating optimization over two subproblems, enabling efficient synchronization and reduced latency. Evaluations on CIFAR-10 with VGG architectures show ESFL achieves significant time efficiency improvements over FL, SL, and SFL while maintaining competitive accuracy. This makes privacy-preserving distributed learning more practical in real-world, heterogeneous wireless networks.

Abstract

Federated learning (FL) allows multiple parties (distributed devices) to train a machine learning model without sharing raw data. How to effectively and efficiently utilize the resources on devices and the central server is a highly interesting yet challenging problem. In this paper, we propose an efficient split federated learning algorithm (ESFL) to take full advantage of the powerful computing capabilities at a central server under a split federated learning framework with heterogeneous end devices (EDs). By splitting the model into different submodels between the server and EDs, our approach jointly optimizes user-side workload and server-side computing resource allocation by considering users' heterogeneity. We formulate the whole optimization problem as a mixed-integer non-linear program, which is an NP-hard problem, and develop an iterative approach to obtain an approximate solution efficiently. Extensive simulations have been conducted to validate the significantly increased efficiency of our ESFL approach compared with standard federated learning, split learning, and splitfed learning.

Figures (7)

• Figure 1: The architecture of splitfed learning (SFL) system.
• Figure 2: The split training procedure, where all selected users simultaneously transmit activation data and labels to the server, and the server sends back the corresponding activation difference.
• Figure 3: This figure illustrates the federated aggregation, where the previous global ML model is $\mathcal{\textbf{W}}^{r}$, the aggregated global ML model is $\mathcal{\textbf{W}}^{r}_{*}$ and the updated global ML model is $\mathcal{\textbf{W}}^{r+1}$
• Figure 4: Testing accuracy and loss over CIFAR-10 testing dataset for FL, SFL, ESFL and SL using three different NN (VGG13, VGG16 and VGG19). Fair comparison are guaranteed by the required training rounds to achieve the convergence threshold.
• Figure 5: Cut layer distributions (user-side workloads allocation) of three NNs under four different resource limitations using ESFL algorithm.