Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LoRA Soups: Merging LoRAs for Practical Skill Composition Tasks

Akshara Prabhakar, Yuanzhi Li, Karthik Narasimhan, Sham Kakade, Eran Malach, Samy Jelassi

TL;DR

The paper tackles skill composition in LLMs by showing that merging task-specific LoRAs can outperform data mixing for combining two skills. It introduces Learnable Concatenation (CAT), a layer-wise, trainable weighting scheme for LoRA merges, and demonstrates its superiority over DATA-MIX and other merging baselines across hard math-word problems, QABot scenarios, and reading comprehension on technical documents. A key finding is a super-linear improvement when combining math and code skills, indicating true compositional synergy. Limitations include the current focus on two skills and variability in robustness to prompt formats, with future work aimed at scaling to more skills and developing more robust merging strategies for larger skill sets.

Abstract

Low-Rank Adaptation (LoRA) is a popular technique for parameter-efficient fine-tuning of Large Language Models (LLMs). We study how different LoRA modules can be merged to achieve skill composition -- testing the performance of the merged model on a target task that involves combining multiple skills, each skill coming from a single LoRA. This setup is favorable when it is difficult to obtain training data for the target task and when it can be decomposed into multiple skills. First, we identify practically occurring use-cases that can be studied under the realm of skill composition, e.g. solving hard math-word problems with code, creating a bot to answer questions on proprietary manuals or about domain-specialized corpora. Our main contribution is to show that concatenation of LoRAs (CAT), which optimally weights LoRAs that were individually trained on different skills, outperforms existing model- and data- merging techniques; for instance on math-word problems, CAT beats these methods by an average of 43% and 12% respectively. Thus, this paper advocates model merging as an efficient way to solve compositional tasks and underscores CAT as a simple, compute-friendly and effective procedure. To our knowledge, this is the first work demonstrating the superiority of model merging over data mixing for binary skill composition tasks. Code and data are available at https://github.com/aksh555/LoRA-Soups

LoRA Soups: Merging LoRAs for Practical Skill Composition Tasks

TL;DR

The paper tackles skill composition in LLMs by showing that merging task-specific LoRAs can outperform data mixing for combining two skills. It introduces Learnable Concatenation (CAT), a layer-wise, trainable weighting scheme for LoRA merges, and demonstrates its superiority over DATA-MIX and other merging baselines across hard math-word problems, QABot scenarios, and reading comprehension on technical documents. A key finding is a super-linear improvement when combining math and code skills, indicating true compositional synergy. Limitations include the current focus on two skills and variability in robustness to prompt formats, with future work aimed at scaling to more skills and developing more robust merging strategies for larger skill sets.

Abstract

Low-Rank Adaptation (LoRA) is a popular technique for parameter-efficient fine-tuning of Large Language Models (LLMs). We study how different LoRA modules can be merged to achieve skill composition -- testing the performance of the merged model on a target task that involves combining multiple skills, each skill coming from a single LoRA. This setup is favorable when it is difficult to obtain training data for the target task and when it can be decomposed into multiple skills. First, we identify practically occurring use-cases that can be studied under the realm of skill composition, e.g. solving hard math-word problems with code, creating a bot to answer questions on proprietary manuals or about domain-specialized corpora. Our main contribution is to show that concatenation of LoRAs (CAT), which optimally weights LoRAs that were individually trained on different skills, outperforms existing model- and data- merging techniques; for instance on math-word problems, CAT beats these methods by an average of 43% and 12% respectively. Thus, this paper advocates model merging as an efficient way to solve compositional tasks and underscores CAT as a simple, compute-friendly and effective procedure. To our knowledge, this is the first work demonstrating the superiority of model merging over data mixing for binary skill composition tasks. Code and data are available at https://github.com/aksh555/LoRA-Soups

Paper Structure

This paper contains 48 sections, 13 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Setting
  4. Skill composition
  5. Hard math-word problems.
  6. QABot on proprietary manuals.
  7. Reading comprehension on technical documents.
  8. Merging methods
  9. Data mixing (DATA-MIX).
  10. Concatenation of LoRAs.
  11. Linear merging of LoRAs.
  12. Experimental details
  13. Skill Finetuning.
  14. DATA-MIX.
  15. CAT.
  16. ...and 33 more sections

Figures (13)

  • Figure 1: Method and performance overview of Learnable Concatenation (CAT).
  • Figure 2: An example of solving hard word-math problem using code.
  • Figure 3: An example of question-answering task based on Dungeons & Dragons game manual.
  • Figure 4: An example of the reading comprehension task in the biomedical domain.
  • Figure 5: CAT vs. DATA solving a GSM-Hard problem. DATA makes frequent coding errors.
  • ...and 8 more figures