Large Language Models as Model Organisms for Human Associative Learning

TL;DR

The paper probes how associative learning reshapes internal representations by treating large language models as model organisms. By adapting a neuroscience learning paradigm to in-context learning across six open-source LLMs, it demonstrates a non-monotonic, memory-like differentiation pattern during Consolidation, aligned with the Non-Monotonic Plasticity Hypothesis. It further shows that vocabulary interference—global competition with prior knowledge—modulates differentiation, increasing it particularly for mid- to high-similarity pairs. These findings position LLMs as scalable, general computational models for investigating principles of memory reorganization and for generating testable hypotheses about brain learning dynamics.

Abstract

Associative learning--forming links between co-occurring items--is fundamental to human cognition, reshaping internal representations in complex ways. Testing hypotheses on how representational changes occur in biological systems is challenging, but large language models (LLMs) offer a scalable alternative. Building on LLMs' in-context learning, we adapt a cognitive neuroscience associative learning paradigm and investigate how representations evolve across six models. Our initial findings reveal a non-monotonic pattern consistent with the Non-Monotonic Plasticity Hypothesis, with moderately similar items differentiating after learning. Leveraging the controllability of LLMs, we further show that this differentiation is modulated by the overlap of associated items with the broader vocabulary--a factor we term vocabulary interference, capturing how new associations compete with prior knowledge. We find that higher vocabulary interference amplifies differentiation, suggesting that representational change is influenced by both item similarity and global competition. Our findings position LLMs not only as powerful tools for studying representational dynamics in human-like learning systems, but also as accessible and general computational models for generating new hypotheses about the principles underlying memory reorganization in the brain.

Figures (16)

• Figure 1: (A) Hypotheses about representational changes in humans. Competing theories propose different patterns of representational change as a function of pair similarity before learning: Hebbian learning predicts integration, pattern separation predicts differentiation, and the Non-Monotonic Plasticity Hypothesis (NMPH) predicts a U-shaped curve, with high differentiation at moderate similarity levels prior to learning. (B) Schematic of our adapted associative learning task for LLMs. Given repeated in-context presentations of a token pair $(x,y)$, the LLM learns to predict the associated token $y$. We measure representational change by computing the difference in cosine similarity between hidden representations of the pair before and after learning. This setup, inspired by a neuroscience paradigm wammes2022increasing, enables us to examine whether similar dynamics of representational restructuring emerge during in-context learning. (C) Illustration of low and high vocabulary interference in the model's representational space. In the low vocabulary interference case (top, yellow), the target token $y$ is dissimilar to most other tokens, resulting in less competition from alternative completions when paired with $x$. In the high interference case (bottom, orange), the pairing $xy$ is highly similar to many other possible token pairings, increasing competition and representational pressure to differentiate the learned association from potential distractors during learning.
• Figure 2: Accuracy and representational changes during learning. (a) Models generally show three phases of learning: encoding, where accuracy steeply increases; consolidation, where accuracy stabilizes; and forgetting, where accuracy declines. To compare across models with different phase lengths, the x-axis is normalized: repetitions within each phase are linearly scaled to fixed intervals ($0-1$ for encoding, $1-2$ for consolidation, $2-3$ for forgetting), allowing phase-aligned trends to be visualized despite variability in learning dynamics. (b) The U-shaped differentiation pattern, characteristic of the Non-Monotonic Plasticity Hypothesis, is observed only during consolidation (red). Asterisks ($\ast$) indicate groups that remain significant after Benjamini--Yekutieli correction for multiple comparisons across similarity groups and phases ($p<0.05$).
• Figure 3: Effect of vocabulary interference on representational change across different pair similarity groups. (a) Results for our original token pairs, sampled uniformly with respect to pair similarity before learning (x-axis). We observe a consistent integration trend for low-similarity pairs and a shift toward differentiation for mid-similarity pairs ($0.6$–$0.7$). High-similarity pairs show more heterogeneous behavior, where low interference tends to promote integration, while higher interference tends to yield differentiation. (b) Results for extended token pairs, sampled uniformly over pair similarity before learning (x-axis) and vocabulary interference level (colored lines). Higher vocabulary interference consistently leads to more differentiation, especially for mid- and high-similarity groups. These results suggest that while pairwise similarity is a key driver of differentiation, vocabulary interference amplifies this effect.
• Figure 4: Log-scale heatmap showing the joint distribution of token pairs across pairwise similarity before learning (x-axis) and vocabulary interference (y-axis) in the representative vocabulary subset $\tilde{\mathcal{V}}$. Subplots (a–f) correspond to individual models; subplot (g) aggregates results across all models.
• Figure 5: Log-scale heatmap showing the joint distribution of token pairs across pairwise similarity before learning (x-axis) and vocabulary interference (y-axis) after uniformly sampling of $10$ items per pairwise similarity $\times$ vocabulary interference bin. Subplots (a–f) correspond to individual models; subplot (g) aggregates results across all models.