LLMs Can Get "Brain Rot"!

TL;DR

The paper tests the LLM Brain Rot Hypothesis: continual pre-training on junk web text degrades LLM cognition across reasoning, long-context understanding, and safety. Using two orthogonal junk-data definitions (M1: engagement-based, M2: semantic-quality), it shows non-trivial declines (e.g., $g>0.3$) and dose-response effects as junk proportion increases, with ARC-Challenge and RULER-CWE illustrating large performance drops. The analysis identifies thought-skipping as a primary failure mode and finds that mitigation via instruction tuning or reflective strategies only partially restores performance, revealing persistent representational drift. The findings argue for training-time safety through data curation and cognitive-health checks, highlighting the need for stronger data-quality controls in continual pre-training.

Abstract

We propose and test the LLM Brain Rot Hypothesis: continual exposure to junk web text induces lasting cognitive decline in large language models (LLMs). To causally isolate data quality, we run controlled experiments on real Twitter/X corpora, constructing junk and reversely controlled datasets via two orthogonal operationalizations: M1 (engagement degree) and M2 (semantic quality), with matched token scale and training operations across conditions. Contrary to the control group, continual pre-training of 4 LLMs on the junk dataset causes non-trivial declines (Hedges' $g>0.3$) on reasoning, long-context understanding, safety, and inflating "dark traits" (e.g., psychopathy, narcissism). The gradual mixtures of junk and control datasets also yield dose-response cognition decay: for example, under M1, ARC-Challenge with Chain Of Thoughts drops $74.9 \rightarrow 57.2$ and RULER-CWE $84.4 \rightarrow 52.3$ as junk ratio rises from $0\%$ to $100\%$. Error forensics reveal several key insights. First, we identify thought-skipping as the primary lesion: models increasingly truncate or skip reasoning chains, explaining most of the error growth. Second, partial but incomplete healing is observed: scaling instruction tuning and clean data pre-training improve the declined cognition yet cannot restore baseline capability, suggesting persistent representational drift rather than format mismatch. Finally, we discover that the popularity, a non-semantic metric, of a tweet is a better indicator of the Brain Rot effect than the length in M1. Together, the results provide significant, multi-perspective evidence that data quality is a causal driver of LLM capability decay, reframing curation for continual pretraining as a \textit{training-time safety} problem and motivating routine "cognitive health checks" for deployed LLMs.

Figures (11)

• Figure 1: Outline of our work: (i) Inspired by the concept of Brain Rot, we establish the hypothesis of LLM Brain Rot; (ii) We construct junk and control data from Twitter/X posts for intervention; (iii) We benchmark four different cognitive functions of the intervened LLMs; (iv) We analyze the results to identify the failure modes caused by the brain rot; and (v) Brain rot is persistent after various mitigation.
• Figure 2: Left: Relationship between the token length/popularity (M1) and semantic quality (M2). $r$ represents the Point-Biserial correlation. Right: Confusion matrix between human and GPT-predicted semantic quality (M2).
• Figure 3: Effective sizes of the proposed intervention. The dark gray/light gray/white areas indicate trivial/non-trivial small/medium effects, respectively. $\downarrow$ indicates the smaller values are preferred. Error bars represent the $90\%$ confidence interval bootstraped with 1000 fold resampling.
• Figure 4: Demonstrations of desired COT and failure modes in answering questions from ARC.
• Figure 5: Failure categorization of COT reasoning on ARC Challenge.