From Associations to Activations: Comparing Behavioral and Hidden-State Semantic Geometry in LLMs

TL;DR

This work investigates whether a large language model's hidden-state semantic geometry can be recovered from observable behavior using psycholinguistic paradigms. By collecting 17.5M+ trials across eight instruction-tuned transformers on a shared 5,000-word noun vocabulary and applying Representational Similarity Analysis, the authors compare behavior-derived semantics to layerwise hidden-state representations under multiple extraction strategies. They find forced-choice behavior aligns substantially more with hidden-state geometry than free association, and that held-out-words ridge regression shows behavior adds predictive power beyond lexical baselines and cross-model consensus, indicating recoverable internal semantic structure from behavior alone. The results highlight the importance of task design and cross-model commonalities for probing internal representations, with practical implications for interpretability and reproducibility in evaluating semantic knowledge in LLMs.

Abstract

We investigate the extent to which an LLM's hidden-state geometry can be recovered from its behavior in psycholinguistic experiments. Across eight instruction-tuned transformer models, we run two experimental paradigms -- similarity-based forced choice and free association -- over a shared 5,000-word vocabulary, collecting 17.5M+ trials to build behavior-based similarity matrices. Using representational similarity analysis, we compare behavioral geometries to layerwise hidden-state similarity and benchmark against FastText, BERT, and cross-model consensus. We find that forced-choice behavior aligns substantially more with hidden-state geometry than free association. In a held-out-words regression, behavioral similarity (especially forced choice) predicts unseen hidden-state similarities beyond lexical baselines and cross-model consensus, indicating that behavior-only measurements retain recoverable information about internal semantic geometry. Finally, we discuss implications for the ability of behavioral tasks to uncover hidden cognitive states.

Figures (15)

• Figure 1: Conceptual overview. For a shared vocabulary $\mathcal{V}$, we (i) extract layer-$\ell$ word representations to form a hidden-state similarity matrix $\mathbf{S}^{\mathrm{hid}}_\ell$, and (ii) run behavioral association tasks (forced choice/free association) to build a cue--response matrix $\mathbf{B}$ and behavioral similarity $\mathbf{S}^{\mathrm{beh}}$. RSA correlates the pairwise similarities in $\mathbf{S}^{\mathrm{hid}}_\ell$ and $\mathbf{S}^{\mathrm{beh}}$ to quantify behavior--activation alignment.
• Figure 2: Behavioral paradigms and derived semantic geometries. Left (forced choice): given a cue word $w_i$ and a candidate set $c_i$, the model selects a fixed number of output words $o_i$, producing a cue--response count matrix $\mathbf{B}^{\mathrm{FC}}$. Right (free association): given $w_i$ alone, the model generates multiple output words $o_i$, yielding $\mathbf{B}^{\mathrm{FA}}$. From the count matrix, we produce similarity matrices $\mathbf{S}^{\mathrm{FC}}$ and $\mathbf{S}^{\mathrm{FA}}$ by cosine similarity between rows. The diagram shows $|c_i|=4$ for FC and $|o_i|=4$ for FA, while our experiments use $|c_i|=16$ for FC and $|o_i|=5$ for FA.
• Figure 3: Summary of RSA and neighborhood-overlap results (means across models). Panel a (left) compares multiple reference geometries: (a1) mean RSA Pearson correlation as a function of layer, and (a2) mean nearest-neighbor overlap (NN@$k$) as a function of neighborhood size $k$ (log scale). Panel b (right) focuses on behavioral references and compares extraction strategies: (b1) layerwise RSA for PPMI-weighted forced-choice similarity $\mathbf{S}^{\mathrm{FC}}$ and (b2) layerwise RSA for PPMI-weighted free-association similarity $\mathbf{S}^{\mathrm{FA}}$.
• Figure 4: Representational similarity analysis between model hidden-state similarity and behavior-derived semantic geometries. Each panel corresponds to a model and contains two sub-heatmaps comparing hidden-state similarity to PPMI-weighted forced-choice ($\mathbf{S}^{\mathrm{FC}}$, left) and PPMI-weighted free-association ($\mathbf{S}^{\mathrm{FA}}$, right) behavioral embeddings. Rows indicate the embedding extraction strategy (Averaged, Meaning, Task (FC), Task (FA)), and columns indicate layerwise correlations (min, max, mean across layers).
• Figure 5: Ridge regression performance for predicting hidden-state similarity from behavioral and lexical features across eight models. Bold values show $R^2$ for the full model (behavioral+FastText+BERT+cross-model consensus); parenthetical values show the FastText+BERT+cross-model consensus baseline. Rows indicate the embedding extraction strategy (Averaged, Meaning, Task (FC), Task (FA)), and columns indicate layerwise correlations (min, max, mean across layers).