Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

QEFT: Quantization for Efficient Fine-Tuning of LLMs

Changhun Lee, Jun-gyu Jin, Younghyun Cho, Eunhyeok Park

TL;DR

This study proposes a new lightweight technique called Quantization for Efficient Fine-Tuning (QEFT), which accelerates both inference and fine-tuning, is supported by robust theoretical foundations, offers high flexibility, and maintains good hardware compatibility.

Abstract

With the rapid growth in the use of fine-tuning for large language models (LLMs), optimizing fine-tuning while keeping inference efficient has become highly important. However, this is a challenging task as it requires improvements in all aspects, including inference speed, fine-tuning speed, memory consumption, and, most importantly, model quality. Previous studies have attempted to achieve this by combining quantization with fine-tuning, but they have failed to enhance all four aspects simultaneously. In this study, we propose a new lightweight technique called Quantization for Efficient Fine-Tuning (QEFT). QEFT accelerates both inference and fine-tuning, is supported by robust theoretical foundations, offers high flexibility, and maintains good hardware compatibility. Our extensive experiments demonstrate that QEFT matches the quality and versatility of full-precision parameter-efficient fine-tuning, while using fewer resources. Our code is available at https://github.com/xvyaward/qeft.

QEFT: Quantization for Efficient Fine-Tuning of LLMs

TL;DR

This study proposes a new lightweight technique called Quantization for Efficient Fine-Tuning (QEFT), which accelerates both inference and fine-tuning, is supported by robust theoretical foundations, offers high flexibility, and maintains good hardware compatibility.

Abstract

With the rapid growth in the use of fine-tuning for large language models (LLMs), optimizing fine-tuning while keeping inference efficient has become highly important. However, this is a challenging task as it requires improvements in all aspects, including inference speed, fine-tuning speed, memory consumption, and, most importantly, model quality. Previous studies have attempted to achieve this by combining quantization with fine-tuning, but they have failed to enhance all four aspects simultaneously. In this study, we propose a new lightweight technique called Quantization for Efficient Fine-Tuning (QEFT). QEFT accelerates both inference and fine-tuning, is supported by robust theoretical foundations, offers high flexibility, and maintains good hardware compatibility. Our extensive experiments demonstrate that QEFT matches the quality and versatility of full-precision parameter-efficient fine-tuning, while using fewer resources. Our code is available at https://github.com/xvyaward/qeft.

Paper Structure

This paper contains 33 sections, 1 theorem, 7 equations, 8 figures, 11 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Weight-only Quantization of LLMs
  4. Parameter-efficient Fine-tuning (PEFT)
  5. Quantization-aware PEFT
  6. Outlier-aware Weight Quantization
  7. Weak Column Tuning
  8. Detailed Overview of QEFT
  9. Data Structure and Quantization Process
  10. Offline Global Reordering
  11. GPU Acceleration Kernel for QEFT
  12. Efficient Backward Computation
  13. Optimal Weak Column Selection
  14. Advanced Application: PEFT Merging
  15. Experiments
  16. ...and 18 more sections

Key Result

Theorem 1

where $\nabla L(\theta^0)_{:,i}$ is the $i$-th channel of $\nabla L(\theta^0)$, then

Figures (8)

  • Figure 1: Pareto-front comparison of PEFT methods. (Left): Average few-shot accuracy after fine-tuning vs. inference speed. (Right): Fine-tuning time vs. inference speed. For more details, please see \ref{['sec:exp_setting']}.
  • Figure 2: The overview of adaptable quantization includes: (a) QLoRA, (b) OWQ, and (c) the proposed QEFT with OGR. $k=4$ case as an example.
  • Figure 3: (a) Weak column indices at all transformer blocks in the Llama-2 7B model, where "attn" indicates input activation of the attention block and "ffn" indicates input activation of the feed-forward block. (b) An overview of the offline global reordering.
  • Figure 4: Visualization of the reduction in computation and memory usage of QEFT in linear layers.
  • Figure 5: Channel-wise sensitivity and the average magnitude of the gradient in Llama-2 7B model. The yellow box indicates the selected weak columns for $k=128$ case.
  • ...and 3 more figures

Theorems & Definitions (1)

  • Theorem