CG-FedLLM: How to Compress Gradients in Federated Fune-tuning for Large Language Models

TL;DR

The paper tackles privacy and communication bottlenecks in federated fine-tuning of large language models by introducing CG-FedLLM, a gradient compression framework that places an AutoEncoder on clients (encoder) and on the server (decoder). It couples this with two training strategies, TGAP for pretraining the AutoEncoder and FAF for federated fine-tuning, to preserve essential gradient information while reducing uplink traffic. Empirical results on LLaMA-7B, Alpaca-7B, and Qwen-7B across C-Eval and MMLU show improved or competitive performance relative to centralized and LoRA-based baselines, along with formal privacy guarantees via DP. The work provides insights into gradient signal preservation, compression-induced robustness, and practical deployment considerations for privacy-aware, communication-efficient FedLLMs.

Abstract

The success of current Large-Language Models (LLMs) hinges on extensive training data that is collected and stored centrally, called Centralized Learning (CL). However, such a collection manner poses a privacy threat, and one potential solution is Federated Learning (FL), which transfers gradients, not raw data, among clients. Unlike traditional networks, FL for LLMs incurs significant communication costs due to their tremendous parameters. This study introduces an innovative approach to compress gradients to improve communication efficiency during LLM FL, formulating the new FL pipeline named CG-FedLLM. This approach integrates an encoder on the client side to acquire the compressed gradient features and a decoder on the server side to reconstruct the gradients. We also developed a novel training strategy that comprises Temporal-ensemble Gradient-Aware Pre-training (TGAP) to identify characteristic gradients of the target model and Federated AutoEncoder-Involved Fine-tuning (FAF) to compress gradients adaptively. Extensive experiments confirm that our approach reduces communication costs and improves performance (e.g., average 3 points increment compared with traditional CL- and FL-based fine-tuning with LlaMA on a well-recognized benchmark, C-Eval). This improvement is because our encoder-decoder, trained via TGAP and FAF, can filter gradients while selectively preserving critical features. Furthermore, we present a series of experimental analyses focusing on the signal-to-noise ratio, compression rate, and robustness within this privacy-centric framework, providing insight into developing more efficient and secure LLMs.

Figures (12)

• Figure 1: Design Variants of FL Approaches for LLMs.
• Figure 2: CG-FedLLM: Compress Gradients in Federated Fune-tuning for Large Language Models. The AutoEncoder $\mathcal{V}$ involved in CG-FedLLM is completed by TGAP (left), and the illustration of utilizing $\mathcal{V}$ in FL is implemented by FAF (right).
• Figure 3: Convergence of training and test loss in TGAP using different architectures: 1D-CNN (left), zero-block U-former, one-block U-former, and two-block U-former (right). The x-axis shows iteration steps; the y-axis shows log MSE reconstruction loss. Blue and yellow lines denote training and test loss, respectively.
• Figure 4: Architectures of AutoEncoder with different building blocks. From top to bottom are 1D CNN abrahamyan2021learned, ResNet he2016deep, and Uformer wang2022uformer respectively.
• Figure 5: Distribution of maximum values (left) and minimum values (right) of matrices $\mathbf{A}$'s and $\mathbf{B}$'s from the iteration 1 to iteration 20. Y axis represents the training epoch, X axis denotes the index of the matrices $\mathbf{A}$ and $\mathbf{B}$ to compress ($2 \times 4 \times 32=128$ dimension).

Theorems & Definitions (5)

• Definition 1: Client-level Differential Privacy geyer2017differentially
• Definition 2: Clipping
• Definition 3: Noising
• Lemma 1: Sensitivity of $\tilde{\mathcal{M}}$
• Theorem 2: Gaussian Differential Privact bu2019deep