Communication-Pipelined Split Federated Learning for Foundation Model Fine-Tuning in UAV Networks

TL;DR

This work addresses the bottleneck of stragglers in split federated learning (SFL) for fine‑tuning foundation models on UAVs by introducing communication‑pipelined SFL (CPSFL). CPSFL combines downlink gradient transmission (GT) priority scheduling with intra‑round asynchronous training to maximize overlap between communication and computation, reducing per‑round latency in downlink‑dominant UAV networks. A joint optimization of split point selection (SPS) and computing/communication resource allocation (CCRA) is solved via an attention‑based deep reinforcement learning (DRL) framework that leverages prior round information and UAV trajectories. Simulation results show CPSFL outperforms parallel GT benchmarks and fixed schemes, and the DRL‑based CPSFL nearly matches the best fixed SPS, highlighting its practical potential for real‑time FM fine‑tuning in dynamic UAV environments.

Abstract

Deploying foundation models (FMs) on uncrewed aerial vehicles (UAVs) promises broad ``low-altitude economy'' applications. Split federated learning (SFL)-based fine-tuning leverages distributed data while keeping raw data local and reduces client-side burden by partitioning the model between client and server. However, the per-round training latency is dominated by stragglers. Training paradigms featuring parallel gradient transmission (GT) allocate dedicated portions of downlink communication resources to each client. They may leave resources idle and suffer from prolonged GT latency, especially in UAV networks, where the communication latency typically far exceeds the computation latency. To address this, we propose a sequential GT paradigm, where the server dedicates all downlink resources for the current GT. We further propose communication-pipelined SFL (CPSFL), characterized by downlink GT priority scheduling and intra-round asynchronous training. We investigate CPSFL-based LoRA fine-tuning of FMs in UAV networks and formulate an optimization problem to minimize a weighted sum of per-round training latency and worst-case client energy consumption by optimizing the split point selection (SPS) and the computing and communication resource allocation (CCRA) (the uplink bandwidth allocation and the server computing frequency allocation). To solve this problem, we develop an attention-based deep reinforcement learning (DRL) framework, where the base station agent decides the split point and the CCRA in each round by leveraging previous round information, including UAV trajectories. Simulation results show that the proposed DRL-based CPSFL scheme outperforms the parallel GT benchmarks, the ablation variants, the fixed CCRA scheme, while approaching the best fixed-SPS scheme.

Figures (9)

• Figure 1: The timeline of one training round (consisting of two local iterations) for three clients when SFL-PP (top), SFL-SP (middle), or SFL-PS (bottom) is applied. The notations SM, CF, CA, SF, SB, SG, CB, and CM correspond to the eight steps in Fig. \ref{['scenario']}, respectively.
• Figure 2: Communication-pipelined split federated learning for foundation models LoRA fine-tuning in UAV networks.
• Figure 3: The timeline of one training round (consisting of two local iterations) for three clients when the proposed CPSFL and its ablation variants are applied. The notations SM, CF, CA, SF, SB, SG$k$, CB, and CM correspond to the eight steps in Fig. \ref{['scenario']}, respectively.
• Figure 4: Timeline of one training round (consisting of two local iterations) for ten clients when SFL-PP (top), PipeSFL (middle), or CPSFL (bottom) is applied.
• Figure 5: Average per-round latency with different split points $u$ obtained by CPSFL and the benchmarks.

Theorems & Definitions (2)

• proof
• proof