Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations

Ziyao Wang, Zheyu Shen, Yexiao He, Guoheng Sun, Hongyi Wang, Lingjuan Lyu, Ang Li

TL;DR

A new approach called FLORA is introduced that enables federated fine-tuning on heterogeneous LoRA adapters across clients through a novel stacking-based aggregation method that is noise-free and seamlessly supports heterogeneous LoRA adapters.

Abstract

The rapid development of Large Language Models (LLMs) has been pivotal in advancing AI, with pre-trained LLMs being adaptable to diverse downstream tasks through fine-tuning. Federated learning (FL) further enhances fine-tuning in a privacy-aware manner by utilizing clients' local data through in-situ computation, eliminating the need for data movement. However, fine-tuning LLMs, given their massive scale of parameters, poses challenges for clients with constrained and heterogeneous resources in FL. Previous methods employed low-rank adaptation (LoRA) for efficient federated fine-tuning but utilized traditional FL aggregation strategies on LoRA adapters. These approaches led to mathematically inaccurate aggregation noise, reducing fine-tuning effectiveness and failing to address heterogeneous LoRAs. In this work, we first highlight the mathematical incorrectness of LoRA aggregation in existing federated fine-tuning methods. We introduce a new approach called FLORA that enables federated fine-tuning on heterogeneous LoRA adapters across clients through a novel stacking-based aggregation method. Our approach is noise-free and seamlessly supports heterogeneous LoRA adapters. Extensive experiments demonstrate FLORA' s superior performance in both homogeneous and heterogeneous settings, surpassing state-of-the-art methods. We envision this work as a milestone for efficient, privacy-preserving, and accurate federated fine-tuning of LLMs. Our code is available at https://github.com/ATP-1010/FederatedLLM.

FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations

TL;DR

A new approach called FLORA is introduced that enables federated fine-tuning on heterogeneous LoRA adapters across clients through a novel stacking-based aggregation method that is noise-free and seamlessly supports heterogeneous LoRA adapters.

Abstract

The rapid development of Large Language Models (LLMs) has been pivotal in advancing AI, with pre-trained LLMs being adaptable to diverse downstream tasks through fine-tuning. Federated learning (FL) further enhances fine-tuning in a privacy-aware manner by utilizing clients' local data through in-situ computation, eliminating the need for data movement. However, fine-tuning LLMs, given their massive scale of parameters, poses challenges for clients with constrained and heterogeneous resources in FL. Previous methods employed low-rank adaptation (LoRA) for efficient federated fine-tuning but utilized traditional FL aggregation strategies on LoRA adapters. These approaches led to mathematically inaccurate aggregation noise, reducing fine-tuning effectiveness and failing to address heterogeneous LoRAs. In this work, we first highlight the mathematical incorrectness of LoRA aggregation in existing federated fine-tuning methods. We introduce a new approach called FLORA that enables federated fine-tuning on heterogeneous LoRA adapters across clients through a novel stacking-based aggregation method. Our approach is noise-free and seamlessly supports heterogeneous LoRA adapters. Extensive experiments demonstrate FLORA' s superior performance in both homogeneous and heterogeneous settings, surpassing state-of-the-art methods. We envision this work as a milestone for efficient, privacy-preserving, and accurate federated fine-tuning of LLMs. Our code is available at https://github.com/ATP-1010/FederatedLLM.
Paper Structure (27 sections, 15 equations, 7 figures, 4 tables)

This paper contains 27 sections, 15 equations, 7 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Fine-tuning LLMs with LoRA.
  4. FedIT: Averaging Homogeneous LoRA.
  5. Proposed Method: FLoRA
  6. Stacking-based Noise-free Aggregation
  7. FLoRA: Stacking-based Federated Fine-tuning for Heterogeneous LoRA
  8. Initialization.
  9. Local Fine-tuning.
  10. Stacking-based LoRA Aggregation.
  11. Update Local Models.
  12. Experiments
  13. Experiment Setup
  14. Baselines.
  15. Experiment Results
  16. ...and 12 more sections

Figures (7)

  • Figure 1: The overview of LoRA, FedIT, and our FLoRA. The top row shows how LoRA updates the model in centralized fine-tuning. The middle and bottom rows show the global model updating strategies in FedIT and our FLoRA respectively.
  • Figure 2: Module stacking in FLoRA is a noise-free aggregation for LoRA, while the module averaging in FedIT cannot accurately aggregate the local updates.
  • Figure 3: FLoRA workflow. The local LoRA modules are initialized and optimized each round, and stacked by the server to obtain the global LoRA modules. The global modules are then sent back to clients to update local models.
  • Figure 4: Standalone experiment results. The red bars represent the global model performance and the blue bars represent the local model performance with varying LoRA ranks.
  • Figure 5: The impact of the scaling factor on FLoRA. The x-axis is the scaling factor, and the y-axis represents the MMLU accuracy for (a)-(b) and the MT-bench score for (c)-(d). The results of Llama2 are in Appendix \ref{['sec:appen_experiment']}.
  • ...and 2 more figures