Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Merging Beyond: Streaming LLM Updates via Activation-Guided Rotations

Yuxuan Yao, Haonan Sheng, Qingsong Lv, Han Wu, Shuqi Liu, Zehua Liu, Zengyan Liu, Jiahui Gao, Haochen Tan, Xiaojin Fu, Haoli Bai, Hing Cheung So, Zhijiang Guo, Linqi Song

TL;DR

Streaming Merging is proposed, an innovative model updating paradigm that conceptualizes merging as an iterative optimization process that surpasses the fully converged SFT model and provides a scalable and lightweight framework for model adaptation.

Abstract

The escalating scale of Large Language Models (LLMs) necessitates efficient adaptation techniques. Model merging has gained prominence for its efficiency and controllability. However, existing merging techniques typically serve as post-hoc refinements or focus on mitigating task interference, often failing to capture the dynamic optimization benefits of supervised fine-tuning (SFT). In this work, we propose Streaming Merging, an innovative model updating paradigm that conceptualizes merging as an iterative optimization process. Central to this paradigm is \textbf{ARM} (\textbf{A}ctivation-guided \textbf{R}otation-aware \textbf{M}erging), a strategy designed to approximate gradient descent dynamics. By treating merging coefficients as learning rates and deriving rotation vectors from activation subspaces, ARM effectively steers parameter updates along data-driven trajectories. Unlike conventional linear interpolation, ARM aligns semantic subspaces to preserve the geometric structure of high-dimensional parameter evolution. Remarkably, ARM requires only early SFT checkpoints and, through iterative merging, surpasses the fully converged SFT model. Experimental results across model scales (1.7B to 14B) and diverse domains (e.g., math, code) demonstrate that ARM can transcend converged checkpoints. Extensive experiments show that ARM provides a scalable and lightweight framework for efficient model adaptation.

Merging Beyond: Streaming LLM Updates via Activation-Guided Rotations

TL;DR

Streaming Merging is proposed, an innovative model updating paradigm that conceptualizes merging as an iterative optimization process that surpasses the fully converged SFT model and provides a scalable and lightweight framework for model adaptation.

Abstract

The escalating scale of Large Language Models (LLMs) necessitates efficient adaptation techniques. Model merging has gained prominence for its efficiency and controllability. However, existing merging techniques typically serve as post-hoc refinements or focus on mitigating task interference, often failing to capture the dynamic optimization benefits of supervised fine-tuning (SFT). In this work, we propose Streaming Merging, an innovative model updating paradigm that conceptualizes merging as an iterative optimization process. Central to this paradigm is \textbf{ARM} (\textbf{A}ctivation-guided \textbf{R}otation-aware \textbf{M}erging), a strategy designed to approximate gradient descent dynamics. By treating merging coefficients as learning rates and deriving rotation vectors from activation subspaces, ARM effectively steers parameter updates along data-driven trajectories. Unlike conventional linear interpolation, ARM aligns semantic subspaces to preserve the geometric structure of high-dimensional parameter evolution. Remarkably, ARM requires only early SFT checkpoints and, through iterative merging, surpasses the fully converged SFT model. Experimental results across model scales (1.7B to 14B) and diverse domains (e.g., math, code) demonstrate that ARM can transcend converged checkpoints. Extensive experiments show that ARM provides a scalable and lightweight framework for efficient model adaptation.
Paper Structure (34 sections, 20 equations, 7 figures, 13 tables)

This paper contains 34 sections, 20 equations, 7 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Single-turn Model Merging
  4. Multi-turn Model Merging
  5. Methodology
  6. The Streaming Merging Framework
  7. Simulating Gradient Descent via Activation-guided Rotation: From SFT Perspective
  8. Activation-guided Rotation-aware Model Merging
  9. Experiments
  10. Models
  11. Datasets and Benchmarks
  12. Baselines
  13. Main Results
  14. Further Analyses
  15. Ablation Study on Hyperparameter $\lambda$, Checkpoint Counts, and Batch Size
  16. ...and 19 more sections

Figures (7)

  • Figure 1: Comparison among different merging pipelines. For Streaming Merging, we select SFT as an illustration.
  • Figure 2: Comparison among Different Merging Approaches. $\theta$ denotes model weights. Gray arrows indicate gradient directions during training, evolving in the full parameter space rather than a planar subspace. (a) TA: The merging outcome is arithmetic. (b) Our ARM: we operate in the full parameter space by rotational alignment, rather than limiting merging to a flat manifold, which is more flexible. (c) OPCM for continual merging: Orthogonal projection is not suitable for training dynamics merging, as checkpoints exhibit complementary rather than mutually exclusive relationships.
  • Figure 3: Ablation Study on Hyperparameter $\lambda$
  • Figure 4: Ablation Study on batch size, we analyze the cosine similarity of the weights before and after rotation.
  • Figure 5: Time consumption analysis: SFT vs ARM
  • ...and 2 more figures