STAR: Spectral Truncation and Rescale for Model Merging

TL;DR

STAR proposes Spectral Truncation And Rescale to merge multiple fine-tuned models without data or extra fine-tuning. It truncates task-update matrices in their spectral spaces to reduce conflicts and then rescales to preserve the nuclear norm, enabling stable averaging across tasks. The method is data-free, robust to hyperparameters (notably eta), and shows strong gains across NLP tasks and model sizes, including up to a 4.2% improvement with 12 models on Flan-T5. This approach provides a practical, scalable way to build multi-task models from many PEFT-finetuned components, broadening the applicability of model merging in real-world settings.

Abstract

Model merging is an efficient way of obtaining a multi-task model from several pretrained models without further fine-tuning, and it has gained attention in various domains, including natural language processing (NLP). Despite the efficiency, a key challenge in model merging is the seemingly inevitable decrease in task performance as the number of models increases. In this paper, we propose $\mathbf{S}$pectral $\mathbf{T}$runcation $\mathbf{A}$nd $\mathbf{R}$escale (STAR) that aims at mitigating ``merging conflicts'' by truncating small components in the respective spectral spaces, which is followed by an automatic parameter rescaling scheme to retain the nuclear norm of the original matrix. STAR requires no additional inference on original training data and is robust to hyperparamater choice. We demonstrate the effectiveness of STAR through extensive model merging cases on diverse NLP tasks. Specifically, STAR works robustly across varying model sizes, and can outperform baselines by 4.2$\%$ when merging 12 models on Flan-T5. Our code is publicly available at https://github.com/IBM/STAR.

Figures (7)

• Figure 1: The averaged normalized performance of Flan-T5-base merged models by TIES yadav2024ties, MetaGPT zhou2024metagpt, TALL-masks wang2024localizing, and STAR (this paper).
• Figure 2: An overview of the STAR workflow. When merging two task vectors, $\delta_{1}$ and $\delta_{2}$, (1) STAR transforms both task vectors into their spectral spaces with their singular vectors being the orthogonal basis using singular value decomposition (SVD) (singular values are represented by the length of the arrows), (2) STAR removes redundant dimensions by truncating singular vectors with small singular values, (3) STAR restores the original nuclear norm by rescaling the truncated SVD, and (4) STAR reconstructs the parameters by multiplying components back to form the weight matrices and then perform simple averaging.
• Figure 3: An example of the automatic rank determination by STAR ($\eta=40$) on PIQA's task vector with Flan-T5-large.
• Figure 4: Model merging results on Flan-T5-large and Mistral-7B-Instruct. For all numbers of models merged, we sampled 5 task combinations for Flan-T5 and 3 for Mistral, with the sampled combinations represented by shaded dots and the average depicted by solid lines. While STAR remains a strong model merging method, TIES, TALL-masks and MetaGPT can be more sensitive to model architecture choice.
• Figure 5: The mean and standard deviation of the optimal $\eta$, which yields the best merged model performance, decrease as the number of merged models increases.