Levels of Analysis for Large Language Models

Abstract

Modern artificial intelligence systems, such as large language models, are increasingly powerful but also increasingly hard to understand. Recognizing this problem as analogous to the historical difficulties in understanding the human mind, we argue that methods developed in cognitive science can be useful for understanding large language models. We propose a framework for applying these methods based on the levels of analysis that David Marr proposed for studying information processing systems. By revisiting established cognitive science techniques relevant to each level and illustrating their potential to yield insights into the behavior and internal organization of large language models, we aim to provide a toolkit for making sense of these new kinds of minds.

Figures (6)

• Figure 1: Large language models perform better when they need to produce a high-probability piece of text than when they need to produce a low-probability piece of text, even in deterministic settings where probability should not matter.
• Figure 2: Incoherent probability judgments from humans (a, b) and GPT-4 (c, d). Like human probability judgments (a), GPT-4’s judgments systematically deviate from zero when combined into probabilistic identities (c). When repeatedly queried about the same event, the mean-variance relationship of probability judgments follows an inverted-U shape for both humans (b) and GPT-4 (d). Human data are adapted from zhu2020bayesian, GPT-4 results are from zhu2024incoherent.
• Figure 3: Patterns of behavior consistent with parallel processing in vision-language models (VLMs). (a) VLMs show highly accurate "pop-out" search for distinctive visual targets but exhibit degraded performance in conjunction search as the number of distractors increases. (b) VLMs exhibit a "subitizing limit" in numerical estimation.
• Figure 4: Exploring the sensory representations of large language models with similarity judgments. (a) For musical pitch, both humans and LLMs show a decrease in judged similarity with increases in the interval between tones, but also show an increase at tones an octave apart (a full similarity matrix for GPT-3 is shown inset). As a consequence, both human and LLM similarities are best captured by helical solutions when converted into spatial representations by multidimensional scaling. (b) Two-dimensional multidimensional scaling solutions for vocal consonants and colors for GPT-4 similarity matrices, showing that LLMs can reproduce patterns seen in human representations despite never having had direct experience of sound or color.
• Figure 5: Large language models such as GPT-4 have been trained to identify situations that involve expressing explicit biases. However, it is possible to construct simple prompts that reveal that they still have strong implicit biases, as reflected in their associations between words. These implicit biases have consequences for their downstream decisions as well.