Repetitions are not all alike: distinct mechanisms sustain repetition in language models

TL;DR

The paper investigates why language models repeatedly generate identical sequences by disentangling multiple underlying mechanisms and their development during training. It contrasts natural repetition with in-context learning (ICL) induced repetition, using developmental trajectories, attention-head activations, and confidence (entropy) analyses to reveal distinct circuitry and dynamics. The findings show that ICL repetition relies on a dedicated, progressively specialized attention-head circuit (with late MLP involvement), whereas natural repetition arises early without a defined circuitry and often focuses on low-information tokens, suggesting a fallback strategy. These results demonstrate that superficially similar repetition behaviors in LLMs originate from qualitatively different internal processes, with implications for mitigation and robust generation across tasks.

Abstract

Large Language Models (LLMs) can sometimes degrade into repetitive loops, persistently generating identical word sequences. Because repetition is rare in natural human language, its frequent occurrence across diverse tasks and contexts in LLMs remains puzzling. Here we investigate whether behaviorally similar repetition patterns arise from distinct underlying mechanisms and how these mechanisms develop during model training. We contrast two conditions: repetitions elicited by natural text prompts with those induced by in-context learning (ICL) setups that explicitly require copying behavior. Our analyses reveal that ICL-induced repetition relies on a dedicated network of attention heads that progressively specialize over training, whereas naturally occurring repetition emerges early and lacks a defined circuitry. Attention inspection further shows that natural repetition focuses disproportionately on low-information tokens, suggesting a fallback behavior when relevant context cannot be retrieved. These results indicate that superficially similar repetition behaviors originate from qualitatively different internal processes, reflecting distinct modes of failure and adaptation in language models.

Figures (8)

• Figure 1: Proportion of repetitions across checkpoints in the ICL (left) and natural (right) settings. At each training step, the same prompts are re-input to the model. Flows indicate whether they remain in the repeating (colored) or non-repeating (light grey) category across steps. Early-emerging repetitions are largely preserved across checkpoints in the natural setting while ICL is late emerging.
• Figure 2: Attention head contrast between outputing a repeating token--above 0--or non repeating--below 0. On the left are multiple steps of ICL training, showing progressive specialisation of specific heads in either directions. On the right is the final training step for the natural dataset. Heads do not contribute either way. Heads with biggest variation across cycle number are put in full opacity. In the legend, we use the layer.head-number format.
• Figure 3: Average attention head focus on different token categories for the two different datasets. Attention focus is reported as a ratio of the relative importance given to tokens compared to their proportion in the original dataset. The leftmost bar on the left indicates that on average, attention heads will assign 9.9 times more attention it would if attention was equally distributed between all tokens in the dataset.
• Figure 4: Percentage of attention weight that shifts to different token categories when we remove all newline tokens from the sequence. While ICL fallbacks on function words primarily, in natural sequences attention heads mostly focus on structural tokens, specific to the tokenizer.
• Figure 5: Entropy of the token probability space for the first token in a cycle, for different number of repeated cycles. Low entropy is when probability mass from the distribution is concentrated in a few tokens. Higher entropy would spread probability mass to more tokens.