Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CAT Merging: A Training-Free Approach for Resolving Conflicts in Model Merging

Wenju Sun, Qingyong Li, Yangli-ao Geng, Boyang Li

TL;DR

The proposed Conflict-Aware Task Merging (CAT Merging), a novel training-free framework that selectively trims conflict-prone components from the task vectors, effectively suppresses knowledge conflicts.

Abstract

Multi-task model merging offers a promising paradigm for integrating multiple expert models into a unified model without additional training. Existing state-of-the-art techniques, such as Task Arithmetic and its variants, merge models by accumulating task vectors -- the parameter differences between pretrained and finetuned models. However, task vector accumulation is often hindered by knowledge conflicts, leading to performance degradation. To address this challenge, we propose Conflict-Aware Task Merging (CAT Merging), a novel training-free framework that selectively trims conflict-prone components from the task vectors. CAT Merging introduces several parameter-specific strategies, including projection for linear weights and masking for scaling and shifting parameters in normalization layers. Extensive experiments on vision, language, and vision-language tasks demonstrate that CAT Merging effectively suppresses knowledge conflicts, achieving average accuracy improvements of up to 2.5% (ViT-B/32) and 2.0% (ViT-L/14) over state-of-the-art methods.

CAT Merging: A Training-Free Approach for Resolving Conflicts in Model Merging

TL;DR

The proposed Conflict-Aware Task Merging (CAT Merging), a novel training-free framework that selectively trims conflict-prone components from the task vectors, effectively suppresses knowledge conflicts.

Abstract

Multi-task model merging offers a promising paradigm for integrating multiple expert models into a unified model without additional training. Existing state-of-the-art techniques, such as Task Arithmetic and its variants, merge models by accumulating task vectors -- the parameter differences between pretrained and finetuned models. However, task vector accumulation is often hindered by knowledge conflicts, leading to performance degradation. To address this challenge, we propose Conflict-Aware Task Merging (CAT Merging), a novel training-free framework that selectively trims conflict-prone components from the task vectors. CAT Merging introduces several parameter-specific strategies, including projection for linear weights and masking for scaling and shifting parameters in normalization layers. Extensive experiments on vision, language, and vision-language tasks demonstrate that CAT Merging effectively suppresses knowledge conflicts, achieving average accuracy improvements of up to 2.5% (ViT-B/32) and 2.0% (ViT-L/14) over state-of-the-art methods.
Paper Structure (36 sections, 1 theorem, 30 equations, 5 figures, 6 tables, 1 algorithm)

This paper contains 36 sections, 1 theorem, 30 equations, 5 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Traditional Multi-Task Learning
  4. Multi-Task Learning through Model Merging
  5. Preliminary
  6. Knowledge Conflict
  7. Methodology
  8. Linear Weight Trimming
  9. Scaling Parameter Trimming
  10. Shifting Parameter Trimming
  11. Experiments
  12. Settings
  13. Comparison Results
  14. Ablation Study
  15. Sensitivity Analysis
  16. ...and 21 more sections

Key Result

Theorem 5.1

Suppose that for any task, the loss function $L$ is $\beta$-Lipschitz continuous with respect to the final network output within the range of model merging. Additionally, assume that each layer $l$ in the network is $\gamma_l$-Lipschitz continuous with respect to the output of layer $(l-1)$ within t

Figures (5)

  • Figure 1: An example of knowledge conflict between two task vectors with differing magnitudes. Magnitude-based algorithms inherently favor the task vector with a larger magnitude (Task 1), leading to the overwriting of knowledge from Task 2.
  • Figure 2: (a) Average accuracy (%) of CAT Merging on eight vision tasks with different numbers of exemplars per task. (b) Average accuracy (%) on eight vision tasks with ViT/L-14 models versus different $\alpha$ (scaling factor in Task Arithmetic, cf. Eq. \ref{['eq:ta']}).
  • Figure 3: Visualization of knowledge conflict on Cars and RESISC45 (i.e., $\Delta L_{\text{Cars}, \text{RESISC45}} + \Delta L_{\text{RESISC45}, \text{Cars}}$) when merging ViT-L/14 models with different merging weights $\alpha_\text{Cars}$ and $\alpha_\text{RESISC45}$.
  • Figure 4: Average accuracy (%) of CAT Merging on eight vision tasks with different values of $\lambda$ (a) and $c$ (b).
  • Figure 5: Visualization of knowledge conflict when merging two ViT-L/14 models.

Theorems & Definitions (3)

  • Theorem 5.1: An Upper Bound on Loss Difference
  • proof
  • proof