From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning

TL;DR

This work asks whether LLMs navigate the same compression-meaning trade-off as humans in conceptual representations. By integrating Rate-Distortion Theory with Information Bottleneck and calibrating against digitized classic cognitive benchmarks, it shows that LLMs broadly align with human category boundaries yet miss fine-grained semantics, while encoder architectures often outperform larger decoders in human alignment. The study reveals a two-phase training dynamic where rapid category formation precedes efficiency-driven reorganization, shifting semantic processing to mid-network layers. Its findings challenge the notion that statistical optimality equates to understanding and argue for models that preserve human-like conceptual inefficiencies to support flexible reasoning and generalization.

Abstract

Humans organize knowledge into compact conceptual categories that balance compression with semantic richness. Large Language Models (LLMs) exhibit impressive linguistic abilities, but whether they navigate this same compression-meaning trade-off remains unclear. We apply an Information Bottleneck framework to compare human conceptual structure with embeddings from 40+ LLMs using classic categorization benchmarks. We find that LLMs broadly align with human category boundaries, yet fall short on fine-grained semantic distinctions. Unlike humans, who maintain ``inefficient'' representations that preserve contextual nuance, LLMs aggressively compress, achieving more optimal information-theoretic compression at the cost of semantic richness. Surprisingly, encoder models outperform much larger decoder models in human alignment, suggesting that understanding and generation rely on distinct representational mechanisms. Training-dynamics analysis reveals a two-phase trajectory: rapid initial concept formation followed by architectural reorganization, during which semantic processing migrates from deep to mid-network layers as the model discovers increasingly efficient, sparser encodings. These divergent strategies, where LLMs optimize for compression and humans for adaptive utility, reveal fundamental differences between artificial and natural intelligence. This highlights the need for models that preserve the conceptual ``inefficiencies'' essential for human-like understanding.

Figures (29)

• Figure 1: Overview of the data generation and analyses. Human data was collected by asking whether an item i (e.g., chair) is a good example of the concept C (furniture). These ratings are aggregated into ranked similarity profiles for each category. Models generate analogous scores using their embeddings. We then compute three metrics: [RQ1] MI to assess category recoverability, [RQ2] Spearman correlation to measure alignment with human internal typicality structure, and [RQ3] a rate-distortion objective capturing the trade-off between representation complexity and meaning preservation.
• Figure 2: LLMs capture categorical boundaries (RQ1 AMI scores) but miss internal geometry (RQ2 Spearman correlations).Left: All 40+ models achieve above-chance AMI with human categories, with encoder architectures (squares, circles, and Xs) matching or exceeding decoder models 100× larger (stars). Results show the layer with peak AMI score per model (see static and mean scores in Figure A.\ref{['fig:static_avg_peak_ami']}). Right: Despite categorical success, models show weak correlations ($\rho < 0.2$ for most) with human typicality judgments, revealing divergent representational strategies. This divergence between capturing boundaries (compression) while missing internal structure (meaning) reveals how LLMs and humans fundamentally differ in their representational strategies. We note that encoder models align more than decoder models, but these correlations are still modest. Computed using the static embeddings, see full results in Tables \ref{['tab:static_spearman_correlations']}-\ref{['tab:peak_spearman_correlations']} in Appendix \ref{['app:exp2']}.
• Figure 3: Divergent optimization strategies: LLMs achieve superior information-theoretic efficiency while humans preserve semantic richness.(a) Encoder models (BERT, ViT, classic embeddings) consistently achieve lower distortion than decoder models at any given complexity level. (b) All LLM-derived clusters achieve lower $\mathcal{L}$ values than human categories (dashed line), indicating more "optimal" compression-distortion balance. Data from rosch1975cognitive.
• Figure 4: Average pooling demonstrates consistent performance across different models, making it the most reliable choice for future research. Each point corresponds to a prompt template applied to a model.
• Figure 5: Average pooling demonstrates the tightest distribution, indicating the highest consistency and reliability across different conditions. Performance Distribution by Pooling Strategy - Box plots showing AMI distribution for each pooling strategy