Language Models Model Language

TL;DR

The paper argues for a radical empiricist, Mańczakian lens to understand language and LLMs, challenging Saussurean signs and Chomskian competence theories. It defines language as the totality of all that is said and written, with frequency of use as the primary organizing principle, treating high-frequency patterns as grammar and viewing meaning as relational within the textual ecosystem. By reframing LLMs as large-scale observers and manipulators of text frequency and analogy, it claims their success empirically validates a text-first, distributional approach to language and offers a practical program for design, evaluation, and interpretation. The authors acknowledge limitations, advocate pragmatic, use-based evaluation over grounding-centric criteria, and discuss ethical, epistemological, and creativity-related questions as avenues for future work.

Abstract

Linguistic commentary on LLMs, heavily influenced by the theoretical frameworks of de Saussure and Chomsky, is often speculative and unproductive. Critics challenge whether LLMs can legitimately model language, citing the need for "deep structure" or "grounding" to achieve an idealized linguistic "competence." We argue for a radical shift in perspective towards the empiricist principles of Witold Mańczak, a prominent general and historical linguist. He defines language not as a "system of signs" or a "computational system of the brain" but as the totality of all that is said and written. Above all, he identifies frequency of use of particular language elements as language's primary governing principle. Using his framework, we challenge prior critiques of LLMs and provide a constructive guide for designing, evaluating, and interpreting language models.