Dynamic Subset Tuning: Expanding the Operational Range of Parameter-Efficient Training for Large Language Models

TL;DR

This work proposes a novel parameter-efficient training method for large language models that adapts models to downstream tasks by optimizing a small subset of the existing model parameters by optimizing a small subset of the existing model parameters.

Abstract

We propose a novel parameter-efficient training (PET) method for large language models that adapts models to downstream tasks by optimizing a small subset of the existing model parameters. Unlike prior methods, this subset is not fixed in location but rather which parameters are modified evolves over the course of training. This dynamic parameter selection can yield good performance with many fewer parameters than extant methods. Our method enables a seamless scaling of the subset size across an arbitrary proportion of the total model size, while popular PET approaches like prompt tuning and LoRA cover only a small part of this spectrum. We match or outperform prompt tuning and LoRA in most cases on a variety of NLP tasks (MT, QA, GSM8K, SuperGLUE) for a given parameter budget across different model families and sizes.

Figures (9)

• Figure 1: Validation set performance of the Gecko and Otter models as a function of the fraction of free parameters relative to full fine-tuning.
• Figure 2: Average scores of T5 models on SuperGLUE compared to full fine-tuning as a function of $\epsilon$.
• Figure 3: Latency increase of applying subset weights on-the-fly before decoding as a function of $\epsilon$. We use the MT test sentence "Wie geht es dir heute?".
• Figure 4: BLEURT scores and the fraction of free parameters per network module on the MT development set (Gecko model, $\epsilon=10^{-6}$). Mean and standard deviation (error bars) across four training runs are shown.
• Figure 5: Training steps per second on TPU v4 chips with a batch size of 32. We use a 2x2x4 topology for the Gecko model and 4x4x4 for Otter and Bison.