Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

From Complex to Simple: Enhancing Multi-Constraint Complex Instruction Following Ability of Large Language Models

Qianyu He, Jie Zeng, Qianxi He, Jiaqing Liang, Yanghua Xiao

TL;DR

The paper tackles the challenge of enabling LLMs to follow complex, multi-constraint instructions by adopting a data-centric framework. It demonstrates that compositional training data (with 3–5 constraints) improves instruction following more than atomic data, especially for smaller models and lower-complexity tasks. A discrimination-based data collection method generates high-quality compositional data, and a contrastive training objective combining Direct Preference Optimization (DPO) with a supervised loss leverages positive and negative samples to boost performance while preserving general ability. Extensive experiments across in-domain, out-of-domain, and adversarial settings show robust improvements and improved training efficiency, underscoring the practical impact of data quality and training strategy on complex instruction following.

Abstract

It is imperative for Large language models (LLMs) to follow instructions with elaborate requirements (i.e. Complex Instructions Following). Yet, it remains under-explored how to enhance the ability of LLMs to follow complex instructions with multiple constraints. To bridge the gap, we initially study what training data is effective in enhancing complex constraints following abilities. We found that training LLMs with instructions containing multiple constraints enhances their understanding of complex instructions, especially those with lower complexity levels. The improvement can even generalize to compositions of out-of-domain constraints. Additionally, we further propose methods addressing how to obtain and utilize the effective training data. Finally, we conduct extensive experiments to prove the effectiveness of our methods in terms of overall performance and training efficiency. We also demonstrate that our methods improve models' ability to follow instructions generally and generalize effectively across out-of-domain, in-domain, and adversarial settings, while maintaining general capabilities.

From Complex to Simple: Enhancing Multi-Constraint Complex Instruction Following Ability of Large Language Models

TL;DR

The paper tackles the challenge of enabling LLMs to follow complex, multi-constraint instructions by adopting a data-centric framework. It demonstrates that compositional training data (with 3–5 constraints) improves instruction following more than atomic data, especially for smaller models and lower-complexity tasks. A discrimination-based data collection method generates high-quality compositional data, and a contrastive training objective combining Direct Preference Optimization (DPO) with a supervised loss leverages positive and negative samples to boost performance while preserving general ability. Extensive experiments across in-domain, out-of-domain, and adversarial settings show robust improvements and improved training efficiency, underscoring the practical impact of data quality and training strategy on complex instruction following.

Abstract

It is imperative for Large language models (LLMs) to follow instructions with elaborate requirements (i.e. Complex Instructions Following). Yet, it remains under-explored how to enhance the ability of LLMs to follow complex instructions with multiple constraints. To bridge the gap, we initially study what training data is effective in enhancing complex constraints following abilities. We found that training LLMs with instructions containing multiple constraints enhances their understanding of complex instructions, especially those with lower complexity levels. The improvement can even generalize to compositions of out-of-domain constraints. Additionally, we further propose methods addressing how to obtain and utilize the effective training data. Finally, we conduct extensive experiments to prove the effectiveness of our methods in terms of overall performance and training efficiency. We also demonstrate that our methods improve models' ability to follow instructions generally and generalize effectively across out-of-domain, in-domain, and adversarial settings, while maintaining general capabilities.
Paper Structure (33 sections, 6 equations, 3 figures, 22 tables)

This paper contains 33 sections, 6 equations, 3 figures, 22 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Instruction Following
  4. Complex Instruction Tuning
  5. Method
  6. Complex Instruction Synthesis
  7. Teacher Correction
  8. Contrastive Method
  9. Experiments
  10. Experiment Setup
  11. Results
  12. Analysis
  13. Data Quality
  14. Constraints Type
  15. Generalization Experiments
  16. ...and 18 more sections

Figures (3)

  • Figure 1: Real-world applications generally involve instructions with multiple constraints (i.e. Complex Instructions), posing challenges for models.
  • Figure 2: The framework of our study. We first study what training data is effective in enhancing complex instruction following abilities via an empirical study. Then, we design a discrimination-based method to address how to obtain the data. Finally, we propose a method for effectively utilizing positive and negative samples obtained.
  • Figure 3: The performance of training efficiency (left) and each constraint type across different training tokens and training steps (right). D and C denote $\text{Ours-LLaMA2-13B}_\text{Discrimination}$ and $\text{Ours-LLaMA2-13B}_\text{Contrastive}$ respectively.