Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Instruction Diversity Drives Generalization To Unseen Tasks

Dylan Zhang, Justin Wang, Francois Charton

TL;DR

The trade-off between the number of instructions the model is trained on and the number of training samples provided for each instruction is investigated and it is observed that the diversity of the instruction set determines generalization.

Abstract

Instruction tuning -- fine-tuning a large language model (LLM) on pairs of instructions and desired outcomes -- is an approach that enables pre-trained language models to perform real-world tasks and follow human instructions. Its practical success depends on the model learning a broader set of instructions than those it was trained on. Yet the factors that determine model generalization to such \emph{unseen tasks} are not well understood. %To understand the driving factors of generalization, In this paper, we experiment with string rewrites, a symbolic task that serves as a building block for Turing complete Markov algorithms while allowing experimental control of "inputs" and "instructions". We investigate the trade-off between the number of instructions the model is trained on and the number of training samples provided for each instruction and observe that the diversity of the instruction set determines generalization. Generalization emerges once a diverse enough set of tasks is provided, even though very few examples are provided for each task. Instruction diversity also ensures robustness with respect to non-uniform distributions of instructions in the training set.

Instruction Diversity Drives Generalization To Unseen Tasks

TL;DR

The trade-off between the number of instructions the model is trained on and the number of training samples provided for each instruction is investigated and it is observed that the diversity of the instruction set determines generalization.

Abstract

Instruction tuning -- fine-tuning a large language model (LLM) on pairs of instructions and desired outcomes -- is an approach that enables pre-trained language models to perform real-world tasks and follow human instructions. Its practical success depends on the model learning a broader set of instructions than those it was trained on. Yet the factors that determine model generalization to such \emph{unseen tasks} are not well understood. %To understand the driving factors of generalization, In this paper, we experiment with string rewrites, a symbolic task that serves as a building block for Turing complete Markov algorithms while allowing experimental control of "inputs" and "instructions". We investigate the trade-off between the number of instructions the model is trained on and the number of training samples provided for each instruction and observe that the diversity of the instruction set determines generalization. Generalization emerges once a diverse enough set of tasks is provided, even though very few examples are provided for each task. Instruction diversity also ensures robustness with respect to non-uniform distributions of instructions in the training set.
Paper Structure (15 sections, 2 equations, 5 figures, 1 table)

This paper contains 15 sections, 2 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Markov Algorithms and rewrite rules
  4. Experiment Results
  5. Instruction Diversity Drives Generalization
  6. Searching and replacing, the roles of No-Ops
  7. Diversity of Instruction Semantics
  8. Impact of instruction Distributions
  9. Generalization Across Input Semantics
  10. Experiments with Pre-Trained Models
  11. Conclusion
  12. Complement on Markov algorithms
  13. Experimental set-up
  14. Model and Training
  15. Data Generation

Figures (5)

  • Figure 1: Illustration of our symbolic tasks in this paper.
  • Figure 2: Generalization versus number of instructions during training.
  • Figure 3: Generalization versus number of instructions during training.
  • Figure 4: Model's performance when trained on the three classes of restricted semantics. Models trained on 500 or less instructions never generalize.
  • Figure 5: Performance of Llama-2 model on the encrypted-rewriting task. We also conducted uniform / non-uniform sub-samplings to half the total sample size at 9000 instructions. Uniform sub-sampling does not harm performance whereas non-uniform subsampling impacts generalization.