Dynamic Fisher-weighted Model Merging via Bayesian Optimization

TL;DR

Dynamic Fisher-weighted Merging (DF-Merge) presents a unified framework that merges fine-tuned language models by assigning scalable coefficients to task vectors and adaptively optimizing them with Bayesian optimization. By integrating parameter-wise Fisher Information into the merge, DF-Merge balances interference across tasks and navigates toward low-loss basins, achieving superior multitask performance without additional training data. The method unifies two prior merging paradigms—Task Arithmetic and Fisher Merging—into a general objective that can be optimized efficiently, requiring only a modest amount of validation data. Empirical results on T5-base and T5-large across six NLP tasks show DF-Merge consistently outperforms strong baselines and approaches the performance of full multitask fine-tuning, highlighting its practicality as a data-efficient, training-free alternative for multitask model merging.

Abstract

The fine-tuning of pre-trained language models has resulted in the widespread availability of task-specific models. Model merging offers an efficient way to create multi-task models by combining these fine-tuned models at the parameter level, without the need for training data or joint training on multiple datasets. Existing merging approaches typically involve scaling the parameters model-wise or integrating parameter importance parameter-wise. Both approaches exhibit their own weaknesses, leading to a notable performance gap compared to multi-task fine-tuning. In this paper, we unify these seemingly distinct strategies into a more general merging framework, and introduce Dynamic Fisher-weighted Merging (DF-Merge). Specifically, candidate models are associated with a set of coefficients that linearly scale their fine-tuned parameters. Bayesian optimization is applied to dynamically adjust these coefficients, aiming to maximize overall performance on validation sets. Each iteration of this process integrates parameter importance based on the Fisher information conditioned by the coefficients. Experimental results show that DF-Merge outperforms strong baselines across models of different sizes and a variety of tasks. Our analysis shows that the effectiveness of DF-Merge arises from the unified view of merging and that near-optimal performance is achievable in a few iterations, even with minimal validation data.

Figures (9)

• Figure 1: Comparison of DF-Merge with primary approaches in model merging. left: leverages parameter importance evaluated at the fine-tuned models. middle: uniformly scales fine-tuned models to alleviate parameter interference. right: DF-Merge optimizes distinct scaling coefficients and incorporates parameter importance evaluated at the scaled models.
• Figure 2: Accuracy (y-axis) of linear inter/extrapolation between the pre-trained model and fine-tuned models with varying coefficients $\lambda$ (x-axis), T5-base.
• Figure 3: An illustration of DF-Merge. (1) The black-box function $f(\cdot)$ takes coefficients as inputs to (a) scale the task vectors, and (c) merges models (yellow star) after (b) accounting for the parameter importance using FI, where a contour depicts the local loss landscape of a specific task. (2) The validation set accuracy $f(\cdot)$ is used by Bayesian optimization to suggest the best guess on the coefficients for the next iteration that improve $f(\cdot)$.
• Figure 4: Bayesian optimization trajectory of DF-Merge (T5-base, UCB). Coefficients (bottom) and the average validation set accuracy (top) are rendered as a function of iterations. The coefficients with the new highest average accuracy up to their corresponding iteration are shown. The red area denotes initial random evaluations.
• Figure 5: Average test set accuracy (%) of DF-Merge with varying ratios of validation samples used, T5-base, based on a single run.