What does it mean to understand language?

TL;DR

This article argues that true language understanding goes beyond parsing and relies on exportation of linguistic representations from the core language system to extra-linguistic brain networks that build mental models, simulate perceptual/motor content, and retrieve world knowledge. It surveys evidence from fMRI and related methods across domains—Theory of Mind, intuitive physics, navigation, perception, motor action, and emotion—to outline a framework in which the language network exports content to specialized regions. The authors distinguish shallow, language-statistics-driven representations from deep understanding that hinges on integrated situation models and memory, and discuss routing versus broadcasting of information as mechanisms for exportation. By articulating testable predictions and linking cognitive neuroscience with AI models, the piece reframes language understanding as distributed across domain-specific systems that coordinate with the core language network, with implications for education and AI alignment.

Abstract

Language understanding entails not just extracting the surface-level meaning of the linguistic input, but constructing rich mental models of the situation it describes. Here we propose that because processing within the brain's core language system is fundamentally limited, deeply understanding language requires exporting information from the language system to other brain regions that compute perceptual and motor representations, construct mental models, and store our world knowledge and autobiographical memories. We review the existing evidence for this hypothesis, and argue that recent progress in cognitive neuroscience provides both the conceptual foundation and the methods to directly test it, thus opening up a new strategy to reveal what it means, cognitively and neurally, to understand language.

Figures (1)

• Figure 1: Hypothesis: Exportation of Information from the Core Language System to Other Brain Regions Is Necessary for Deep Understanding.A) The language system exports information to extra-linguistic systems that support deep understanding of linguistic content. Candidate exportation destinations include regions specialized for thinking about the contents of someone else’s or one’s own mind [16], regions specialized for scene understanding and navigation [19], regions that construct mental models of the physical world [21], regions that represent and process amodal semantic information [23], perceptual and motor regions (e.g., [11-13]), and regions that integrate information into a coherent representation of the situation that is being described [95-96]. These exportation destination regions are visualized with broad masks delineating where the relevant functional regions are found; although these masks may overlap, the functional regions within individuals are distinct. B) The language system supports shallow language understanding. Shallow language understanding entails recognizing individual words and how they go together within the structure of the sentence (i.e., parsing the surface form of the sentence), and then constructing a compositional representation of the sentence’s meaning, perhaps represented in a high-dimensional vector space. Importantly, these shallow representations are only defined with respect to other linguistic elements and patterns of their use that we have encountered during our lifetime of experience with language, making no contact with other aspects of our lived experience. C) Extra-linguistic systems support deep language understanding. They can be engaged both by linguistic and nonlinguistic inputs, which is why we do not consider them part of the language system. During language understanding these regions may augment the shallow representations constructed by the core language system with perceptual or motor features, translate them into a format compatible with probabilistic programing engines or symbolic problem solvers, or transform them in some other way.