The Closeness of In-Context Learning and Weight Shifting for Softmax Regression
Shuai Li, Zhao Song, Yu Xia, Tong Yu, Tianyi Zhou
TL;DR
This work analyzes in-context learning through a softmax-regression lens on Transformer attention, deriving quantitative bounds on how small updates to the model weight or the input data affect the learned predictions. It formalizes that the data transformation induced by a single self-attention layer and by gradient-descent steps on a softmax regression loss are bounded, establishing a close link between gradient updates and self-attention dynamics for regression tasks. The authors introduce a universal bound factor $M = n^{1.5} \exp(10 R^2)$ and show that both weight shifts and sentence-data shifts produce controlled perturbations in the regression output, thereby explaining the observed similarity between in-context learning and explicit optimization. These results deepen the theoretical understanding of in-context learning in LLMs and provide a rigorous connection between weight shifting and data shifting in softmax-based attention mechanisms.
Abstract
Large language models (LLMs) are known for their exceptional performance in natural language processing, making them highly effective in many human life-related or even job-related tasks. The attention mechanism in the Transformer architecture is a critical component of LLMs, as it allows the model to selectively focus on specific input parts. The softmax unit, which is a key part of the attention mechanism, normalizes the attention scores. Hence, the performance of LLMs in various NLP tasks depends significantly on the crucial role played by the attention mechanism with the softmax unit. In-context learning, as one of the celebrated abilities of recent LLMs, is an important concept in querying LLMs such as ChatGPT. Without further parameter updates, Transformers can learn to predict based on few in-context examples. However, the reason why Transformers becomes in-context learners is not well understood. Recently, several works [ASA+22,GTLV22,ONR+22] have studied the in-context learning from a mathematical perspective based on a linear regression formulation $\min_x\| Ax - b \|_2$, which show Transformers' capability of learning linear functions in context. In this work, we study the in-context learning based on a softmax regression formulation $\min_{x} \| \langle \exp(Ax), {\bf 1}_n \rangle^{-1} \exp(Ax) - b \|_2$ of Transformer's attention mechanism. We show the upper bounds of the data transformations induced by a single self-attention layer and by gradient-descent on a $\ell_2$ regression loss for softmax prediction function, which imply that when training self-attention-only Transformers for fundamental regression tasks, the models learned by gradient-descent and Transformers show great similarity.