Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DR-Encoder: Encode Low-rank Gradients with Random Prior for Large Language Models Differentially Privately

Huiwen Wu, Deyi Zhang, Xiaohan Li, Xiaogang Xu, Jiafei Wu, Zhe Liu

TL;DR

This work addresses privacy risks in federated fine-tuning of large language models by proposing DR-Encoder, a gradient encoding framework that uses a random-prior AutoEncoder trained on synthetic gradient statistics and a subsequent Gaussian-DP-based fine-tuning stage. The method achieves end-to-end client-level differential privacy for FedLLM while maintaining competitive utility and drastically reducing communication overhead, quantified via GDP and RDP analyses. Key contributions include the two-stage randomness design, formal privacy proofs, extensive experiments on Qwen and LlaMa with MMLU and C-Eval, and a thorough privacy-utility and communication-efficiency evaluation. The approach advances practical privacy-preserving federated LLM fine-tuning and offers potential extensions to larger vision and multimodal foundation models.

Abstract

The emergence of the Large Language Model (LLM) has shown their superiority in a wide range of disciplines, including language understanding and translation, relational logic reasoning, and even partial differential equations solving. The transformer is the pervasive backbone architecture for the foundation model construction. It is vital to research how to adjust the Transformer architecture to achieve an end-to-end privacy guarantee in LLM fine-tuning. In this paper, we investigate three potential information leakage during a federated fine-tuning procedure for LLM (FedLLM). Based on the potential information leakage, we provide an end-to-end privacy guarantee solution for FedLLM by inserting two-stage randomness. The first stage is to train a gradient auto-encoder with a Gaussian random prior based on the statistical information of the gradients generated by local clients. The second stage is to fine-tune the overall LLM with a differential privacy guarantee by adopting appropriate Gaussian noises. We show the efficiency and accuracy gains of our proposed method with several foundation models and two popular evaluation benchmarks. Furthermore, we present a comprehensive privacy analysis with Gaussian Differential Privacy (GDP) and Renyi Differential Privacy (RDP).

DR-Encoder: Encode Low-rank Gradients with Random Prior for Large Language Models Differentially Privately

TL;DR

This work addresses privacy risks in federated fine-tuning of large language models by proposing DR-Encoder, a gradient encoding framework that uses a random-prior AutoEncoder trained on synthetic gradient statistics and a subsequent Gaussian-DP-based fine-tuning stage. The method achieves end-to-end client-level differential privacy for FedLLM while maintaining competitive utility and drastically reducing communication overhead, quantified via GDP and RDP analyses. Key contributions include the two-stage randomness design, formal privacy proofs, extensive experiments on Qwen and LlaMa with MMLU and C-Eval, and a thorough privacy-utility and communication-efficiency evaluation. The approach advances practical privacy-preserving federated LLM fine-tuning and offers potential extensions to larger vision and multimodal foundation models.

Abstract

The emergence of the Large Language Model (LLM) has shown their superiority in a wide range of disciplines, including language understanding and translation, relational logic reasoning, and even partial differential equations solving. The transformer is the pervasive backbone architecture for the foundation model construction. It is vital to research how to adjust the Transformer architecture to achieve an end-to-end privacy guarantee in LLM fine-tuning. In this paper, we investigate three potential information leakage during a federated fine-tuning procedure for LLM (FedLLM). Based on the potential information leakage, we provide an end-to-end privacy guarantee solution for FedLLM by inserting two-stage randomness. The first stage is to train a gradient auto-encoder with a Gaussian random prior based on the statistical information of the gradients generated by local clients. The second stage is to fine-tune the overall LLM with a differential privacy guarantee by adopting appropriate Gaussian noises. We show the efficiency and accuracy gains of our proposed method with several foundation models and two popular evaluation benchmarks. Furthermore, we present a comprehensive privacy analysis with Gaussian Differential Privacy (GDP) and Renyi Differential Privacy (RDP).

Paper Structure

This paper contains 35 sections, 5 theorems, 12 equations, 5 figures, 7 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Differential Privacy for LLMs
  4. Parameter Efficient Fine-tuning (PEFT)
  5. Methodology
  6. Privacy Goal
  7. Pre-Training with Random Prior
  8. Differentially Private Federated Fine-tuning with LoRA gradients
  9. Theoretical Analysis
  10. Priliminary
  11. Privacy of DR-Encoder
  12. Experiments
  13. Experimental Settings
  14. Foundation Models Improvement (RQ1)
  15. Privacy Accounting via RDP or GDP (RQ2)
  16. ...and 20 more sections

Key Result

Theorem 4.1

dwork2014algorithmic The Gaussian mechanism defined in Definition def:gaussian_mecha preserves $(\epsilon, \delta)-$differential privacy.

Figures (5)

  • Figure 1: Privacy loss accumulation through RDP. The label denotes the accumulated privacy loss computed by RDP.
  • Figure 2: MMLU Performance Analysis across four disciplines. The y-axis shows the evaluation score, while the x-axis shows the privacy budget. The yellow line denotes FedCG, the green line denotes DR-Encoder, and the blue line denotes LoRA.
  • Figure 3: Architecture of AutoEncoder with Uformer wang2022uformer
  • Figure 4: Gradient Dynamical Visualization of Layer $\{0,1,2,29,30,31\}$.
  • Figure 5: Gradient Dynamic Visualization for Layer $\{4,5,6,26,27,28\}$.

Theorems & Definitions (10)

  • Definition 4.1
  • Definition 4.2
  • Theorem 4.1
  • Lemma 4.1
  • Lemma 4.2: Privacy per iteration
  • Theorem 4.2: Gaussian Differential Privacy bu2019deep
  • proof
  • proof
  • Theorem 8.1: Central Limit Theorem for GDP bu2019deep
  • proof