Knowledge Grafting of Large Language Models

TL;DR

GraftLLM addresses cross-capability transfer between heterogeneous LLMs by encoding source-model capabilities as compact, modular SkillPacks within a target model. It introduces a module-aware adaptive compression strategy that prunes and decomposes deltas per module (embedding, MLP, attention) and applies adaptive quantization, enabling effective knowledge fusion and forget-free continual learning. The approach combines supervised fine-tuning with direct preference optimization and uses a routing mechanism to fuse multiple SkillPacks while isolating task-specific knowledge to mitigate interference. Across ten benchmarks and varied fusion and continual-learning setups, GraftLLM demonstrates strong performance, often matching or exceeding fully finetuned baselines at significantly reduced storage costs and with the ability to unload or detoxify as needed.

Abstract

Cross-capability transfer is a key challenge in large language model (LLM) research, with applications in multi-task integration, model compression, and continual learning. Recent works like FuseLLM and FuseChat have demonstrated the potential of transferring multiple model capabilities to lightweight models, enhancing adaptability and efficiency, which motivates our investigation into more efficient cross-capability transfer methods. However, existing approaches primarily focus on small, homogeneous models, limiting their applicability. For large, heterogeneous models, knowledge distillation with full-parameter fine-tuning often overlooks the student model's intrinsic capacity and risks catastrophic forgetting, while PEFT methods struggle to effectively absorb knowledge from source LLMs. To address these issues, we introduce GraftLLM, a novel method that stores source model capabilities in a target model with SkillPack format. This approach preserves general capabilities, reduces parameter conflicts, and supports forget-free continual learning and model fusion. We employ a module-aware adaptive compression strategy to compress parameter updates, ensuring efficient storage while maintaining task-specific knowledge. The resulting SkillPack serves as a compact and transferable knowledge carrier, ideal for heterogeneous model fusion and continual learning. Experiments across various scenarios demonstrate that GraftLLM outperforms existing techniques in knowledge transfer, knowledge fusion, and forget-free learning, providing a scalable and efficient solution for cross-capability transfer. The code is publicly available at: https://github.com/duguodong7/GraftLLM.

Figures (14)

• Figure 1: Comparison of explicit knowledge fusion methods for heterogeneous LLMs on MT-Bench, including parameter size analysis.
• Figure 2: A comprehensive comparison of implicit knowledge fusion methods for heterogeneous LLMs across multiple benchmarks.
• Figure 3: Comparision of knowledge distillation and knowledge grafting in various scenarios.
• Figure 4: Overview of GraftLLM. GraftLLM transfers capabilities across heterogeneous LLMs and extracts them into compact SkillPacks, facilitating efficient knowledge fusion and continual learning.
• Figure 5: Comparison of parameter efficiency and MMLU performance across different methods for LLM capability transfer with SFT.