Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The unreasonable effectiveness of pattern matching

Gary Lupyan, Blaise Agüera y Arcas

TL;DR

The paper addresses how to conceptualize large language models (LLMs) by showing that they function as powerful pattern-matching engines whose capabilities expand with data and scale, rather than as simple databases or blurry representations of the web. Through Jabberwocky-like texts and The Gostak, it demonstrates that LLMs can recover meaning by leveraging syntactic patterns and contextual cues, effectively deblurring degraded inputs via structural fingerprints. It connects these findings to construction grammar, arguing that language consists of patterns ranging from specific constructions to abstract templates, which constrain interpretation. The overarching thesis is that pattern matching is a general cognitive principle that underpins LLM success and broader intelligence, offering a unifying lens for AI, cognition, and the limits of current metaphors like “universal mimic” or “blurry JPEG of the web.”

Abstract

We report on an astonishing ability of large language models (LLMs) to make sense of "Jabberwocky" language in which most or all content words have been randomly replaced by nonsense strings, e.g., translating "He dwushed a ghanc zawk" to "He dragged a spare chair". This result addresses ongoing controversies regarding how to best think of what LLMs are doing: are they a language mimic, a database, a blurry version of the Web? The ability of LLMs to recover meaning from structural patterns speaks to the unreasonable effectiveness of pattern-matching. Pattern-matching is not an alternative to "real" intelligence, but rather a key ingredient.

The unreasonable effectiveness of pattern matching

TL;DR

The paper addresses how to conceptualize large language models (LLMs) by showing that they function as powerful pattern-matching engines whose capabilities expand with data and scale, rather than as simple databases or blurry representations of the web. Through Jabberwocky-like texts and The Gostak, it demonstrates that LLMs can recover meaning by leveraging syntactic patterns and contextual cues, effectively deblurring degraded inputs via structural fingerprints. It connects these findings to construction grammar, arguing that language consists of patterns ranging from specific constructions to abstract templates, which constrain interpretation. The overarching thesis is that pattern matching is a general cognitive principle that underpins LLM success and broader intelligence, offering a unifying lens for AI, cognition, and the limits of current metaphors like “universal mimic” or “blurry JPEG of the web.”

Abstract

We report on an astonishing ability of large language models (LLMs) to make sense of "Jabberwocky" language in which most or all content words have been randomly replaced by nonsense strings, e.g., translating "He dwushed a ghanc zawk" to "He dragged a spare chair". This result addresses ongoing controversies regarding how to best think of what LLMs are doing: are they a language mimic, a database, a blurry version of the Web? The ability of LLMs to recover meaning from structural patterns speaks to the unreasonable effectiveness of pattern-matching. Pattern-matching is not an alternative to "real" intelligence, but rather a key ingredient.
Paper Structure (8 sections, 2 figures, 3 tables)

This paper contains 8 sections, 2 figures, 3 tables.

Table of Contents

  1. What sort of thing is a large language model?
  2. From Jabberwocky to The Gostak
  3. Language as a set of patterns
  4. Pattern matching all the way down
  5. Acknowledgments
  6. Appendix
  7. Example of an LLM playing The Gostak.
  8. An example of an LLM recovering text that has had all content words completely masked

Figures (2)

  • Figure 1: As a rough metric of translation success, we computed similarity between the embeddings (OpenAI text-embedding-3-large) of the original text and LLM "translations" of the Jabberwockified versions. Plotted are the similarity values (1=identical) of the cases described in the text, alongside a distribution of translations of a variety of 150 250-word text passages spanning fiction, podcast transcripts, and TV/movie scripts.
  • Figure 2: (A). This word appears to be too blurred to read, but becomes readable when placed in context. (B.) Examine the image in B upside-down, then look again at the (upright) image shown in A.