The Inherent Limits of Pretrained LLMs: The Unexpected Convergence of Instruction Tuning and In-Context Learning Capabilities

TL;DR

The study investigates whether instruction-tuned LLMs offer fundamentally new capabilities beyond base models prompted with in-context examples. It demonstrates a significant correlation between instruction-tuned performance and base-model in-context performance across model families and scales, suggesting that both are bounded by priors from pretraining data, with instruction tuning contributing via its own data. Through analyses of confounds like prompt complexity, semantic labeling, and tuning-task composition, the authors show instruction-tuned models do not unlock qualitatively new task solvability beyond base models. These findings challenge claims of emergent or novel abilities from instruction tuning and have implications for evaluating, deploying, and governing LLMs in practice.

Abstract

Large Language Models (LLMs), trained on extensive web-scale corpora, have demonstrated remarkable abilities across diverse tasks, especially as they are scaled up. Nevertheless, even state-of-the-art models struggle in certain cases, sometimes failing at problems solvable by young children, indicating that traditional notions of task complexity are insufficient for explaining LLM capabilities. However, exploring LLM capabilities is complicated by the fact that most widely-used models are also "instruction-tuned" to respond appropriately to prompts. With the goal of disentangling the factors influencing LLM performance, we investigate whether instruction-tuned models possess fundamentally different capabilities from base models that are prompted using in-context examples. Through extensive experiments across various model families, scales and task types, which included instruction tuning 90 different LLMs, we demonstrate that the performance of instruction-tuned models is significantly correlated with the in-context performance of their base counterparts. By clarifying what instruction-tuning contributes, we extend prior research into in-context learning, which suggests that base models use priors from pretraining data to solve tasks. Specifically, we extend this understanding to instruction-tuned models, suggesting that their pretraining data similarly sets a limiting boundary on the tasks they can solve, with the added influence of the instruction-tuning dataset.

Figures (8)

• Figure 1: Our main results for LLaMA-2 13B demonstrate a significant correlation between the performance of base models prompted with in-context examples (and their variants) and that of an instruction-tuned model. Base-D is the zero-shot base model, close to the random baseline. This correlation is additionally dependant on the task clusters present in the instruction-tuning data. This is particularly visible in the isolated comparison of the FLAN and Gold Pipeline models (left). Plots showing generalisation across model families and scale are in the Appendix (\ref{['app:fig:llama_7B_hyp1']}, \ref{['app:fig:mistral_7B_hyp1']}).
• Figure 2: The tasks used in our experiments, grouped into clusters following wei2022finetuned. Dark-colour backgrounds indicate that the training set of the corresponding task was used in instruction-tuning our models. The tasks with a white background were not seen during training. Tasks with a red border are those used for testing in the supplementary experiments in Section \ref{['subsec:instruction-tuning-tasks']}.
• Figure 3: Overall performance trend of our Llama-2 13B FLAN model across three clusters of tasks, with standard deviation across 3 seeds. The x-axis is truncated to terminate at 40.
• Figure 4: Overall performance trend of our Llama-2 13B FLAN model on QNLI and MNLI-Matched, with standard deviation over 3 seeds. The x-axis is truncated to begin at 20 and terminates at 80.
• Figure 5: Correlation and statistical test results for our main experiments on Llama-2 7B, showing the significant correlation between the performance of base models prompted with in-context examples (and their variants) and that of instruction-tuned models.