DPI: Exploiting Parameter Heterogeneity for Interference-Free Fine-Tuning

TL;DR

This work addresses cross-task interference in supervised fine-tuning of large language models, manifested as the seesaw effect when all parameters are updated uniformly. It introduces Dynamic Parameter Isolation (DPI), a data-driven framework that identifies per-task core parameter regions by measuring update magnitudes during brief, task-specific fine-tuning, then groups tasks by core-region similarity and trains in stages with prior-core parameters frozen. The main contributions are: (1) revealing parameter heterogeneity as a root cause of interference, (2) proposing a three-stage DPI pipeline with core identification, task grouping, and dynamic freezing, and (3) empirical results showing DPI consistently surpasses full SFT and heuristic multi-stage baselines on diverse SFT benchmarks. The findings indicate that freezing task-specific cores preserves prior knowledge while enabling effective learning of new tasks, yielding robust improvements across architectures and tasks, and offering a scalable approach for heterogeneous SFT.

Abstract

Supervised fine-tuning (SFT) is a crucial step for adapting large language models (LLMs) to downstream tasks. However, conflicting objectives across heterogeneous SFT tasks often induce the "seesaw effect": optimizing for one task may degrade performance on others, particularly when model parameters are updated indiscriminately. In this paper, we propose a principled approach to disentangle and isolate task-specific parameter regions, motivated by the hypothesis that parameter heterogeneity underlies cross-task interference. Specifically, we first independently fine-tune LLMs on diverse SFT tasks and identify each task's core parameter region as the subset of parameters exhibiting the largest updates. Tasks with highly overlapping core parameter regions are merged for joint training, while disjoint tasks are organized into different stages. During multi-stage SFT, core parameters acquired in prior tasks are frozen, thereby preventing overwriting by subsequent tasks. To verify the effectiveness of our method, we conducted intensive experiments on multiple public datasets. The results showed that our dynamic parameter isolation strategy consistently reduced data conflicts and achieved consistent performance improvements compared to multi-stage and multi-task tuning baselines.

Figures (2)

• Figure 1: Comparison of fine-tuning methods: (a) timeline of training data input; (b) difference between full-parameter SFT and dynamic parameter isolation fine-tuning.
• Figure 2: Impact of core percentage ($p$).