Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond IID: Optimizing Instruction Learning from the Perspective of Instruction Interaction and Dependency

Hanyu Zhao, Li Du, Yiming Ju, Chengwei Wu, Tengfei Pan

TL;DR

This paper systemically investigates interaction and dependency patterns between different categories of instructions, manages to optimize the instruction set concerning the interaction patterns using a linear programming-based method, and optimize the learning schema of SFT using an instruction dependency taxonomy guided curriculum learning.

Abstract

With the availability of various instruction datasets, a pivotal challenge is how to effectively select and integrate these instructions to fine-tune large language models (LLMs). Previous research mainly focuses on selecting individual high-quality instructions. However, these works overlooked the joint interactions and dependencies between different categories of instructions, leading to suboptimal selection strategies. Moreover, the nature of these interaction patterns remains largely unexplored, let alone optimize the instruction set with regard to them. To fill these gaps, in this paper, we: (1) systemically investigate interaction and dependency patterns between different categories of instructions, (2) manage to optimize the instruction set concerning the interaction patterns using a linear programming-based method, and optimize the learning schema of SFT using an instruction dependency taxonomy guided curriculum learning. Experimental results across different LLMs demonstrate improved performance over strong baselines on widely adopted benchmarks.

Beyond IID: Optimizing Instruction Learning from the Perspective of Instruction Interaction and Dependency

TL;DR

This paper systemically investigates interaction and dependency patterns between different categories of instructions, manages to optimize the instruction set concerning the interaction patterns using a linear programming-based method, and optimize the learning schema of SFT using an instruction dependency taxonomy guided curriculum learning.

Abstract

With the availability of various instruction datasets, a pivotal challenge is how to effectively select and integrate these instructions to fine-tune large language models (LLMs). Previous research mainly focuses on selecting individual high-quality instructions. However, these works overlooked the joint interactions and dependencies between different categories of instructions, leading to suboptimal selection strategies. Moreover, the nature of these interaction patterns remains largely unexplored, let alone optimize the instruction set with regard to them. To fill these gaps, in this paper, we: (1) systemically investigate interaction and dependency patterns between different categories of instructions, (2) manage to optimize the instruction set concerning the interaction patterns using a linear programming-based method, and optimize the learning schema of SFT using an instruction dependency taxonomy guided curriculum learning. Experimental results across different LLMs demonstrate improved performance over strong baselines on widely adopted benchmarks.
Paper Structure (28 sections, 5 equations, 7 figures, 6 tables)

This paper contains 28 sections, 5 equations, 7 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Causal Intervention based Instruction Correlation Analysis and Ability Taxonomy Induction
  3. Instruction Collection and Automatic Ability Tagging System
  4. Causal Intervention based Instruction Correlation Analysis
  5. Experimental Settings
  6. Analysis Results
  7. Causal Intervention based Large Language Model Ability Taxonomy Induction
  8. Experimental Settings
  9. An Empirical Ability Taxonomy of LLM
  10. Category Relationship Guided Instruction Set Optimization
  11. Effect Equivalence-based Category Proportion Optimization
  12. Experimental Settings
  13. Baseline Methods
  14. Results
  15. Ability Dependency Taxonomy Guided Curriculum Instruction Learning
  16. ...and 13 more sections

Figures (7)

  • Figure 1: Framework of our work. Baseline methods selection instructions using quality scores (a). In this paper, we first induce the correlation pattern (b) and dependency taxonomy (d), then optimize the instruction set collection concerning the correlation (c) and dependency taxonomy (e).
  • Figure 2: The effect equivalence coefficients between different categories of instructions.
  • Figure 3: Performance of Llama3-8B and Qwen 1.5-7B fine-tuned on instruction set obtained by EE-CPO and DEITA with different sample sizes.
  • Figure 4: Performance of Llama3-8B and Qwen 1.5-7B fine-tuned on instruction set obtained by DF-CSFT and DEITA with different sample sizes.
  • Figure 5: The effect equivalence coefficients between different categories of instructions derived by Qwen and Llama, respectively.
  • ...and 2 more figures