Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Compatibility of Generative AI and Generative Linguistics

Eva Portelance, Masoud Jasbi

TL;DR

The paper addresses whether generative AI, particularly neural language models (LMs), can be reconciled with and ultimately advance generative linguistics. It argues that LMs function as formal generative models, can support discovery procedures for theory development, and align with Universal Grammar and language acquisition within the Minimalist framework, thereby benefiting both AI and linguistics. The authors advocate grammar-induction LMs as promising discovery tools that can yield descriptive and explanatory adequacy while remaining grounded in Chomsky’s finetuned adequacy criteria. They also emphasize multimodal grounding as a path to richer linguistic theory and more data-efficient language learning in AI systems.

Abstract

In mid-20th century, the linguist Noam Chomsky established generative linguistics, and made significant contributions to linguistics, computer science, and cognitive science by developing the computational and philosophical foundations for a theory that defined language as a formal system, instantiated in human minds or artificial machines. These developments in turn ushered a wave of research on symbolic Artificial Intelligence (AI). More recently, a new wave of non-symbolic AI has emerged with neural Language Models (LMs) that exhibit impressive linguistic performance, leading many to question the older approach and wonder about the the compatibility of generative AI and generative linguistics. In this paper, we argue that generative AI is compatible with generative linguistics and reinforces its basic tenets in at least three ways. First, we argue that LMs are formal generative models as intended originally in Chomsky's work on formal language theory. Second, LMs can help develop a program for discovery procedures as defined by Chomsky's "Syntactic Structures". Third, LMs can be a major asset for Chomsky's minimalist approach to Universal Grammar and language acquisition. In turn, generative linguistics can provide the foundation for evaluating and improving LMs as well as other generative computational models of language.

On the Compatibility of Generative AI and Generative Linguistics

TL;DR

The paper addresses whether generative AI, particularly neural language models (LMs), can be reconciled with and ultimately advance generative linguistics. It argues that LMs function as formal generative models, can support discovery procedures for theory development, and align with Universal Grammar and language acquisition within the Minimalist framework, thereby benefiting both AI and linguistics. The authors advocate grammar-induction LMs as promising discovery tools that can yield descriptive and explanatory adequacy while remaining grounded in Chomsky’s finetuned adequacy criteria. They also emphasize multimodal grounding as a path to richer linguistic theory and more data-efficient language learning in AI systems.

Abstract

In mid-20th century, the linguist Noam Chomsky established generative linguistics, and made significant contributions to linguistics, computer science, and cognitive science by developing the computational and philosophical foundations for a theory that defined language as a formal system, instantiated in human minds or artificial machines. These developments in turn ushered a wave of research on symbolic Artificial Intelligence (AI). More recently, a new wave of non-symbolic AI has emerged with neural Language Models (LMs) that exhibit impressive linguistic performance, leading many to question the older approach and wonder about the the compatibility of generative AI and generative linguistics. In this paper, we argue that generative AI is compatible with generative linguistics and reinforces its basic tenets in at least three ways. First, we argue that LMs are formal generative models as intended originally in Chomsky's work on formal language theory. Second, LMs can help develop a program for discovery procedures as defined by Chomsky's "Syntactic Structures". Third, LMs can be a major asset for Chomsky's minimalist approach to Universal Grammar and language acquisition. In turn, generative linguistics can provide the foundation for evaluating and improving LMs as well as other generative computational models of language.

Paper Structure

This paper contains 5 sections, 1 equation, 6 figures.

Table of Contents

  1. Introduction
  2. On Generative Models, Generative Capacity, and Levels of Adequacy
  3. On Procedures for Linguistic Theory Development
  4. On Learnability, Language Acquisition, and Universal Grammar
  5. Final Discussion

Figures (6)

  • Figure 1: Example of a simple context-free generative grammar and derived sentence, based on a corpus
  • Figure 2: Chomsky-Schützenberger hierarchy
  • Figure 3: The structural ambiguity in the phrase "pancakes or bacon and eggs" captured by a context free grammar (CFG) as two different hierarchical organization of the words: (pancakes or (bacon and eggs) vs. (pancakes or bacon) and eggs), but not captured by a Regular Grammar (RG) that simply concatenates words into the string "pancakes+or+bacon+and+eggs".
  • Figure 4: Procedures for linguistic theory development. Inspired by the original figure from Syntactic Structures, chomsky1957ss.
  • Figure 5: The relative positions of both linguistic and neural network based generative theories of language according to their level of theoretical adequacy and their procedural type.
  • ...and 1 more figures