The Alchemy of Thought: Understanding In-Context Learning Through Supervised Classification

TL;DR

This work probes the mechanisms behind in-context learning (ICL) by comparing LLMs with ICL to supervised classifiers trained on the same demonstrations across six text-classification datasets. It leverages Sentence-BERT to select top-$k$ demonstrations and evaluates three LLMs against $k$NN and LR to determine which classifier best mirrors ICL behavior. The key finding is that, when demonstration relevance is high, ICL behavior aligns more with $k$NN than with gradient-descent-based LR, suggesting the attention mechanism in LLMs behaves similarly to non-parametric retrieval. However, when relevance is low, LLMs outperform these classifiers by leveraging their parametric memory, implying a practical strategy: use $k$NN or LR when relevant demonstrations are available and rely on LLMs’ memory otherwise. The study provides actionable guidance for prompt design and suggests that high-quality, semantically similar demonstrations enable efficient non-parametric approaches for task adaptation.

Abstract

In-context learning (ICL) has become a prominent paradigm to rapidly customize LLMs to new tasks without fine-tuning. However, despite the empirical evidence of its usefulness, we still do not truly understand how ICL works. In this paper, we compare the behavior of in-context learning with supervised classifiers trained on ICL demonstrations to investigate three research questions: (1) Do LLMs with ICL behave similarly to classifiers trained on the same examples? (2) If so, which classifiers are closer, those based on gradient descent (GD) or those based on k-nearest neighbors (kNN)? (3) When they do not behave similarly, what conditions are associated with differences in behavior? Using text classification as a use case, with six datasets and three LLMs, we observe that LLMs behave similarly to these classifiers when the relevance of demonstrations is high. On average, ICL is closer to kNN than logistic regression, giving empirical evidence that the attention mechanism behaves more similarly to kNN than GD. However, when demonstration relevance is low, LLMs perform better than these classifiers, likely because LLMs can back off to their parametric memory, a luxury these classifiers do not have.

Figures (14)

• Figure 1: Performance on six datasets of kNN, LR, GPT-4o-mini, Llama-3.1-8B-Instruct, and Qwen-7B-Chat models for the same $k=10$ examples used by each model. In several datasets, LLMs perform similarly to kNN and LR, but this is not always the case.
• Figure 2: Visualization highlighting the difference in accuracy between kNN (top) and LR (bottom), each compared with three LLMs. Each vertical block is a different dataset; each row is a setting with a different number of ICL examples (from $1$ to $30$). The difference is also encoded with a color gradient, where green means lower difference (higher similarity) and red indicates high differences (low similarity).
• Figure 3: Contingency matrices for AG News: Comparison of kNN vs GPT (top-left), kNN vs Llama (middle-left), and kNN vs Qwen (bottom-left) and LR vs GPT (top-right), LR vs Llama (middle-right), and LR vs Qwen (bottom-right) for $k=20$.
• Figure 4: Contingency matrices for News Category: The same configuration follows from Figure \ref{['fig:agnews']}.
• Figure 5: Contingency matrices for AuTexTification: The same configuration follows from Figure \ref{['fig:agnews']}.