Enhancing Large Language Model Performance with Gradient-Based Parameter Selection

TL;DR

The paper tackles redundancy in full-parameter fine-tuning of large language models by introducing Gradient-Mask Tuning (GMT), which uses task-specific gradient magnitudes to selectively update parameters. GMT accumulates gradients over a training interval, selects the top components by $|\Gamma_{ij}|$ with a threshold $T_k$, and updates only the masked parameters, formalized as $\Gamma_{ij} = \frac{1}{N} \sum_{n=1}^N \nabla_{\theta_{ij}} \mathcal{L}(\Theta, \mathcal{B}_n)$ and $\theta_{ij}^{(t+1)} = \theta_{ij}^{(t)} - \eta \cdot \mathcal{M}(\Gamma_{ij}, k)$. Theoretical analysis links gradient magnitude to parameter saliency via $s_{ij} = |\nabla_{\theta_{ij}} \mathcal{L}(\Theta; \mathcal{D})|$ and a constrained optimization framework using $\mathcal{L}_d(\Theta, \lambda) = \mathcal{L}(\Theta) + \lambda^T \| (I - M)\Theta_\Delta \|^2$ to justify sparsity. Empirically, GMT improves performance across code generation, math reasoning, and general-domain tasks under both SFT and DPO, with robustness to mask ratio and efficiency comparable to vanilla SFT, enabling a plug-and-play replacement without architectural changes. The work demonstrates that task-specific gradient information can guide effective, sparse fine-tuning, reducing redundant updates while elevating upper-bound performance on diverse benchmarks.

Abstract

Large language models (LLMs) have revolutionized lots of fields of research. Although it is well-known that fine-tuning is essential for enhancing the capabilities of LLMs, existing research suggests that there is potential redundancy in the fine-tuning process and therefore proposes to update only a subset of parameters. However, these methods fail to leverage the task-specific information to identify important parameters during training. Based on the insight that gradients inherently contain information on task-specific data, we propose Gradient-Mask Tuning (GMT), a method that selectively updates parameters during training based on their gradient information. Specifically, we compute the absolute values of the gradients and apply masking to those with relatively smaller magnitudes. Our empirical results across various tasks demonstrate that GMT not only outperforms traditional fine-tuning methods but also elevates the upper limits of LLM performance. Further analysis indicates that GMT exhibits insensitivity to mask ratio and possesses computational efficiency comparable to vanilla SFT.

Figures (3)

• Figure 1: Illustration of our proposed method GMT, compared with the one-off drop delta parameters approach. The figure on the left delineates the distinction between a trivial drop and a random drop, wherein the trivial drop serves to diminish the redundant updates that arises during the fine-tuning process. Building upon this insight, we refine the training procedure by preferentially updating more significant parameters, as determined by the gradient information pertinent to the task-specific data. This selective updating is operationalized through the implementation of a masking strategy that filters out gradients with smaller absolute values.
• Figure 2: Fine-tuning performance of the proposed GMT with respect to various mask ratios during training in three domains. The training LLMs utilized for code generation task, math reasoning task, and general domain are DeepSeek-Coder-Base-6.7B, Mistral-7B, and Llama2-7B, respectively. The experimental results for the MATH benchmark are presented on the secondary y-axis located on the right side of the figure.
• Figure 3: We employed three strategies for selectively dropping delta parameters at various rates depending on the magnitude (absolute value) of the delta parameters: 1) preferentially dropping salient parameters, 2) preferentially dropping trivial parameters, and 3) dropping parameters randomly.