Low-latency Federated LLM Fine-tuning Over Wireless Networks

TL;DR

This paper tackles latency challenges in federated fine-tuning of large language models over wireless networks by introducing JCPBA, which jointly optimizes client-specific pruning rates $\beta_k$ and bandwidth allocations $B_k$ within an emulator–adapter framework. The method partitions the LLM into an emulator and an adapter, prunes the emulator per client, and updates only the adapter, enabling efficient on-device training while preserving privacy; the optimization is solved with a block coordinate descent approach under practical resource and convergence constraints. Empirical results on Yahoo Answers and GSM8K show that JCPBA achieves at least 40% faster wall-clock fine-tuning and 46% lower communication overhead, while attaining equal or better test loss, and demonstrates robustness to heterogeneous computation capacities. These findings suggest that dynamic, client-aware pruning and bandwidth scheduling can substantially improve the practicality of federated LLM fine-tuning in wireless edge environments.

Abstract

Recently, federated large language models (LLMs) have drawn significant attention thanks to coupled capabilities of LLMs and federated learning (FL) that address privacy concerns in collaborative fine-tuning. However, due to large-scale parameters of LLMs, existing federated LLM fine-tuning frameworks incur significant challenges in resource-constrained clients characterized by heterogeneous computing capabilities and random wireless channels. To address this issue, we propose a joint client-specific pruning and bandwidth allocation (JCPBA) framework for federated LLMs to improve the fine-tuning efficiency over the wireless networks. Specifically, we formulate a fine-tuning latency minimization problem by jointly optimizing pruning rates and bandwidth allocations. Furthermore, we solve this optimization problem using a block coordinate descent method. Extensive experiments on the datasets of Yahoo Answers and GSM8K demonstrate that the proposed framework significantly reduces wall-clock fine-tuning time compared with state-of-the-art baselines and gains equal or lower test loss at the cost of lower computation and communication overhead.

Figures (4)

• Figure 1: The framework of federated LLM fine-tuning.
• Figure 2: The comparison of test loss on Yahoo Answers and GSM8K.
• Figure 3: Overhead comparison across different methods.
• Figure 4: Comparison of fine-tuning latency under different computation heterogeneity levels.