Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HiFT: A Hierarchical Full Parameter Fine-Tuning Strategy

Yongkang Liu, Yiqun Zhang, Qian Li, Tong Liu, Shi Feng, Daling Wang, Yifei Zhang, Hinrich Schütze

TL;DR

HiFT introduces a memory-efficient, optimizer-independent hierarchical fine-tuning strategy that enables full-parameter fine-tuning by updating only one block of layers at a time. By partitioning the model into $k$ groups of size $m$ layers and employing a delayed learning-rate update, HiFT substantially reduces GPU memory consumption for gradients, optimizer states, and residual activations while preserving performance comparable to FPFT and PEFT. Across diverse models and tasks, including RoBERTa, GPT-2, LLaMA-2, and OPT-13B, HiFT achieves competitive accuracy with significant memory savings, enabling full-parameter fine-tuning on resource-constrained hardware and suggesting potential for parallel block updates in the future. The approach demonstrates a practical path to scalable, high-quality fine-tuning of large language models with reduced architectural and memory overheads, leveraging optimizer compatibility and a simple block-wise update scheme.

Abstract

Full-parameter fine-tuning has become the go-to choice for adapting language models (LMs) to downstream tasks due to its excellent performance. As LMs grow in size, fine-tuning the full parameters of LMs requires a prohibitively large amount of GPU memory. Existing approaches utilize zeroth-order optimizer to conserve GPU memory, which can potentially compromise the performance of LMs as non-zero order optimizers tend to converge more readily on most downstream tasks. In this paper, we propose a novel optimizer-independent end-to-end hierarchical fine-tuning strategy, HiFT, which only updates a subset of parameters at each training step. HiFT can significantly reduce the amount of gradients and optimizer state parameters residing in GPU memory at the same time, thereby reducing GPU memory usage. Our results demonstrate that: (1) HiFT achieves comparable performance to parameter-efficient fine-tuning and standard full parameter fine-tuning. (2) HiFT supports various optimizers including AdamW, AdaGrad, SGD, etc. (3) HiFT can save more than 60\% GPU memory compared with standard full-parameter fine-tuning for 7B model. (4) HiFT enables full-parameter fine-tuning of a 7B model on single 48G A6000 with a precision of 32 using the AdamW optimizer, without using any memory saving techniques.

HiFT: A Hierarchical Full Parameter Fine-Tuning Strategy

TL;DR

HiFT introduces a memory-efficient, optimizer-independent hierarchical fine-tuning strategy that enables full-parameter fine-tuning by updating only one block of layers at a time. By partitioning the model into groups of size layers and employing a delayed learning-rate update, HiFT substantially reduces GPU memory consumption for gradients, optimizer states, and residual activations while preserving performance comparable to FPFT and PEFT. Across diverse models and tasks, including RoBERTa, GPT-2, LLaMA-2, and OPT-13B, HiFT achieves competitive accuracy with significant memory savings, enabling full-parameter fine-tuning on resource-constrained hardware and suggesting potential for parallel block updates in the future. The approach demonstrates a practical path to scalable, high-quality fine-tuning of large language models with reduced architectural and memory overheads, leveraging optimizer compatibility and a simple block-wise update scheme.

Abstract

Full-parameter fine-tuning has become the go-to choice for adapting language models (LMs) to downstream tasks due to its excellent performance. As LMs grow in size, fine-tuning the full parameters of LMs requires a prohibitively large amount of GPU memory. Existing approaches utilize zeroth-order optimizer to conserve GPU memory, which can potentially compromise the performance of LMs as non-zero order optimizers tend to converge more readily on most downstream tasks. In this paper, we propose a novel optimizer-independent end-to-end hierarchical fine-tuning strategy, HiFT, which only updates a subset of parameters at each training step. HiFT can significantly reduce the amount of gradients and optimizer state parameters residing in GPU memory at the same time, thereby reducing GPU memory usage. Our results demonstrate that: (1) HiFT achieves comparable performance to parameter-efficient fine-tuning and standard full parameter fine-tuning. (2) HiFT supports various optimizers including AdamW, AdaGrad, SGD, etc. (3) HiFT can save more than 60\% GPU memory compared with standard full-parameter fine-tuning for 7B model. (4) HiFT enables full-parameter fine-tuning of a 7B model on single 48G A6000 with a precision of 32 using the AdamW optimizer, without using any memory saving techniques.
Paper Structure (29 sections, 1 theorem, 12 equations, 6 figures, 14 tables, 1 algorithm)

This paper contains 29 sections, 1 theorem, 12 equations, 6 figures, 14 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Full-Parameter Fine-tuning
  4. Parameter-Efficient Fine-tuning
  5. Memory-Efficient Fine-tuning
  6. Approach
  7. Notation
  8. Hierarchical Training
  9. Experiments
  10. Results
  11. Prompt results
  12. Instruction Fine-tuning
  13. No prompt results
  14. Memory Efficiency
  15. Wallclock Time Efficiency
  16. ...and 14 more sections

Key Result

Theorem 3

(HiFT generalization bound) Under Assumption assumption:1 and assumption:2, we have the following generalization bound for HiFT: where $\theta^{*}$ denotes the parameter with the best test performance, $\Tilde{\theta}^{(i)}$ is in the space of $\beta_{i} \circ \theta_{pre}$ and $\theta^{(i)*}$ denotes the parameter with the best test performance when only changing the subset parameter $\beta_{i}

Figures (6)

  • Figure 1: Schematic diagram of our HiFT. group represents the grouping operation of the layers. bottom2up, top2down and random are training strategies. Gray indicates that the corresponding parameters are in the frozen state, and brown indicates that the corresponding parameters are in the activated state. $k$ is the number of groups, $n$ is the number of layers of the given model, and BP denotes parameter update through back propagation.
  • Figure 2: Category-wise scores of different fine-tuning methods on MT-bench. The detailed results are shown in Table \ref{['tab:r-mt']} (Appendix \ref{['sec:results']}).
  • Figure 3: Loss curves of OPT-13B on different datasets. The parameter $m$ of HiFT is set to 1.
  • Figure 4: The left shows the performance of HiFT of RoBERTa$_\text{base}$ under B2U, T2D and RAN strategies, respectively. The right shows the performance of HiFT of RoBERTa$_\text{base}$ under different grouping settings, where $m$ is the number of layers in each group.
  • Figure 5: RoBERTa results on different fine-tuning strategies. We report accuracy metrics for the SST-2, QNLI, QQP, MRPC and RTE, mean accuracy for MNLI, spearman coefficient for STS-B and matthews correlation coefficient for CoLA. The $m$ of HiFT is set to 1. B2U, T2D and RAN are bottom2up, top2down and random strategies.
  • ...and 1 more figures

Theorems & Definitions (1)

  • Theorem 3