Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

It's Morphing Time: Unleashing the Potential of Multiple LLMs via Multi-objective Optimization

Bingdong Li, Zixiang Di, Yanting Yang, Hong Qian, Peng Yang, Hao Hao, Ke Tang, Aimin Zhou

TL;DR

A novel approach for addressing the multi-objective optimization problem in large language model merging via black-box multi-objective optimization algorithms via an automated optimization approach named MM-MO, which leverages multi-objective optimization algorithms to autonomously search for optimal merging configurations across various tasks, alleviating the need for human intervention.

Abstract

In this paper, we introduce a novel approach for addressing the multi-objective optimization problem in large language model merging via black-box multi-objective optimization algorithms. The goal of model merging is to combine multiple models, each excelling in different tasks, into a single model that outperforms any of the individual source models. However, model merging faces two significant challenges: First, existing methods rely heavily on human knowledge or intuition. Second, it's difficult to obtain the great model merging configuration in limited evaluations. To address these challenges, we formalize model merging as a multi-objective optimization problem and propose an automated optimization approach named MM-MO. This method leverages multi-objective optimization algorithms to autonomously search for optimal merging configurations across various tasks, alleviating the need for human intervention. In MM-MO, a weak-to-strong method is employed to enhance the acquisition function, allowing previously evaluated superior configurations to guide the search for new ones. Meanwhile, Fisher information is applied to screen these configurations, increasing the possibility of identifying high-quality merging configuration. Additionally, we designed a sparsity metric as an additional optimization objective to enhance the model's generalization performance across different tasks. We conducted comprehensive experiments with other mainstream model merging methods, demonstrating that the proposed MM-MO algorithm is competitive and effective in achieving high-quality model merging.

It's Morphing Time: Unleashing the Potential of Multiple LLMs via Multi-objective Optimization

TL;DR

A novel approach for addressing the multi-objective optimization problem in large language model merging via black-box multi-objective optimization algorithms via an automated optimization approach named MM-MO, which leverages multi-objective optimization algorithms to autonomously search for optimal merging configurations across various tasks, alleviating the need for human intervention.

Abstract

In this paper, we introduce a novel approach for addressing the multi-objective optimization problem in large language model merging via black-box multi-objective optimization algorithms. The goal of model merging is to combine multiple models, each excelling in different tasks, into a single model that outperforms any of the individual source models. However, model merging faces two significant challenges: First, existing methods rely heavily on human knowledge or intuition. Second, it's difficult to obtain the great model merging configuration in limited evaluations. To address these challenges, we formalize model merging as a multi-objective optimization problem and propose an automated optimization approach named MM-MO. This method leverages multi-objective optimization algorithms to autonomously search for optimal merging configurations across various tasks, alleviating the need for human intervention. In MM-MO, a weak-to-strong method is employed to enhance the acquisition function, allowing previously evaluated superior configurations to guide the search for new ones. Meanwhile, Fisher information is applied to screen these configurations, increasing the possibility of identifying high-quality merging configuration. Additionally, we designed a sparsity metric as an additional optimization objective to enhance the model's generalization performance across different tasks. We conducted comprehensive experiments with other mainstream model merging methods, demonstrating that the proposed MM-MO algorithm is competitive and effective in achieving high-quality model merging.
Paper Structure (37 sections, 5 equations, 7 figures, 15 tables, 2 algorithms)

This paper contains 37 sections, 5 equations, 7 figures, 15 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Model Merging
  4. Multi-objective Optimization
  5. Problem Formulation
  6. Methodology
  7. Overview
  8. Model Merging via Multi-objective Optimization
  9. Pseudocode of MM-MO
  10. Enhanced Acquisition Strategy
  11. Weak-to-strong Method
  12. Fisher Information
  13. Sparsity Optimization Objective
  14. Experimental Setup
  15. Source Models
  16. ...and 22 more sections

Figures (7)

  • Figure 1: Illustration of merged model performance differences when formalizing the model merging configuration search as a single-objective vs. multi-objective optimization problem. Multi-objective optimization achieves a more balanced performance across code and math capabilities.
  • Figure 2: An illustration of formalizing model merging as a multi-objective optimization problem, with three source models and three evaluation tasks.
  • Figure 3: An illustration of automated model merging with multi-objective optimization (MM-MO).
  • Figure 4: Performance of single and merged LLMs on different categories of the BBH dataset.
  • Figure 5: Time overhead comparison between MM-MO and EMM under non-parallel evaluation.
  • ...and 2 more figures

Theorems & Definitions (1)

  • Definition 1: Pareto dominance yu1974cone