Rectifying Demonstration Shortcut in In-Context Learning

TL;DR

The paper identifies a Demonstration Shortcut in in-context learning, where LLMs rely on pre-trained semantic priors in demonstrations rather than learning new input–label mappings. It introduces In-Context Calibration (ICC), a demonstration-aware calibration method that estimates per-sample semantic priors from in-context examples and from shuffled word representations, then uses these priors to rectify predictions. Across 27 classification datasets and three model families (GPT, OPT, Llama2), ICC consistently improves both Original ICL Task performance and Task Learning (where labels are semantically unrelated or symbolized), with notable gains in challenging NLI tasks and large-scale models. The approach remains effective after instruction tuning and scales to models above 50B, suggesting broad applicability for enabling LLMs to learn new mappings from demonstrations. Overall, ICC provides a practical, demonstration-level calibration technique to mitigate semantic priors and enhance task learning in diverse LLM settings.

Abstract

Large language models (LLMs) are able to solve various tasks with only a few demonstrations utilizing their in-context learning (ICL) abilities. However, LLMs often rely on their pre-trained semantic priors of demonstrations rather than on the input-label relationships to proceed with ICL prediction. In this work, we term this phenomenon as the 'Demonstration Shortcut'. While previous works have primarily focused on improving ICL prediction results for predefined tasks, we aim to rectify the Demonstration Shortcut, thereby enabling the LLM to effectively learn new input-label relationships from demonstrations. To achieve this, we introduce In-Context Calibration, a demonstration-aware calibration method. We evaluate the effectiveness of the proposed method in two settings: (1) the Original ICL Task using the standard label space and (2) the Task Learning setting, where the label space is replaced with semantically unrelated tokens. In both settings, In-Context Calibration demonstrates substantial improvements, with results generalized across three LLM families (OPT, GPT, and Llama2) under various configurations.

Figures (10)

• Figure 1: The overall illustration of the Demonstration Shortcut. In a zero-shot setting, an LLM predicts the test article to the world label. With the first demonstration set, the LLM predicts the business label through ICL. However, with the second demonstration set — which has the same label order but different semantics in the examples — the LLM predicts the sports label. GPT-J is used for these experiments. See Appendix \ref{['appendix:full_description']} for a full description of the demonstrations.
• Figure 2: Averaged Macro F1 scores for OPT (Top), GPT (Medium), and Llama2 (Bottom) across Sentiment, NLI, and Detection Tasks. The left the left three columns depict the performance on the Original ICL Task. The right three columns plot the Task Learning scores. In both graphs, the x-axis represents the model size.
• Figure 3: Averaged Macro F1 scores for Llama2-Chat across Sentiment, NLI, and Detection Tasks. The left three columns depict the performance of the Original ICL Task. The right three columns plot the Task Learning scores. In both graphs, the x-axis represents the model size.
• Figure 4: Averaged Macro F1 scores across 27 classification tasks for over 50B scale LLMs. The left graphs depict performance in the Original ICL Task, while the right graphs plot task learning scores. In both sets of graphs, the x-axis denotes the model type. Full details of the data-type scores are provided in Appendix \ref{['appendix:more_results']}.
• Figure 5: Averaged Macro F1 scores for the GPT model are presented across 27 classification tasks, each featuring a permuted label space. The x-axis represents the model size. Results for the OPT and Llama2 models are provided in Appendix \ref{['appendix:more_results']}.