Comparative Analysis of Different Efficient Fine Tuning Methods of Large Language Models (LLMs) in Low-Resource Setting

TL;DR

The paper addresses the challenge of efficiently fine-tuning large language models in low-resource settings by systematically comparing vanilla fine-tuning, pattern-based fine-tuning, adaptive fine-tuning, LoRA-based parameter-efficient fine-tuning, and context distillation on MNLI and COLA using OPT models. It employs a rigorous few-shot framework with multiple seeds and consistent hyper-parameters to evaluate in-domain and out-of-domain generalization across two model sizes. Key findings show that context distillation frequently yields superior generalization, PBFT can underperform especially on OOD data, and LoRA and adaptive fine-tuning offer competitive but not consistently superior results relative to full fine-tuning, highlighting clear resource-performance trade-offs. The work provides practical guidance for practitioners on choosing fine-tuning strategies based on memory, compute, and data constraints, with context distillation emerging as a promising approach for efficient generalization.

Abstract

In the domain of large language models (LLMs), arXiv:2305.16938 showed that few-shot full-model fine-tuning -- namely Vanilla Fine Tuning (FT) and Pattern-Based Fine Tuning (PBFT) --, and In-Context Learning (ICL) generalize similarly on Out-Of-Domain (OOD) datasets, but vary in terms of task adaptation. However, they both pose challenges, especially in term of memory requirements. In this paper, we further try to push the understanding of different fine-tuning strategies for LLM and aim to bring a myriad of these on the same pedestal for an elaborate comparison with full-model fine-tuning on two diverse datasets. To that end, we conducted a series of experiments, beginning with state-of-the-art methods like vanilla fine-tuning and Pattern-Based Fine-Tuning (PBFT) on pre-trained models across two datasets, COLA and MNLI. We then investigate adaptive fine-tuning and the efficiency of LoRA adapters in a few-shot setting. Finally, we also compare an alternative approach that has gained recent popularity -- context distillation -- with the vanilla FT and PBFT with and without few-shot setup. Our findings suggest that these alternative strategies that we explored can exhibit out-of-domain generalization comparable to that of vanilla FT and PBFT. PBFT under-performs Vanilla FT on out-of-domain (OOD) data, emphasizing the need for effective prompts. Further, our adaptive-fine tuning and LoRA experiments perform comparable or slightly worse than the standard fine-tunings as anticipated, since standard fine-tunings involve tuning the entire model. Finally, our context distillation experiments out-perform the standard fine-tuning methods. These findings underscore that eventually the choice of an appropriate fine-tuning method depends on the available resources (memory, compute, data) and task adaptability.

Figures (11)

• Figure 1: Weight update step (right) with and (left) without with LoRA adapter, Figure courtesy: sebraschka2024
• Figure 2: Vanilla FT accuracy for OPT 125M and OPT 350M on (left) COLA and (right) MNLI for different samples
• Figure 3: PBFT accuracy for OPT 125M and OPT 350M on (left) COLA and (right) MNLI for different samples
• Figure 4: Adaptive FT accuracy for OPT 125M on COLA for different few-shot samples
• Figure 5: Accuracy on COLA for various few shot samples averaged across ranks of LoRA layers