Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Uniform Information Density and Syntactic Reduction: Revisiting $\textit{that}$-Mentioning in English Complement Clauses

Hailin Hao, Elsi Kaiser

TL;DR

This study revisits Uniform Information Density (UID) as a driver of syntactic reduction by examining that-mentioning in English complement clauses using the CANDOR corpus and neural language models. It compares information density estimates from verb subcategorization probabilities with contextual word embeddings derived from GPT-2 Small, finding that embedding-based density better accounts for that-mentioning after controlling for verb-specific effects. The embedding-based approach explains additional variance over traditional lexical predictors and remains predictive even when verb identity is modeled as a random effect, whereas verb-based density largely disappears under the same treatment. The results support UID in naturalistic speech but also show that high-level structural cues captured by embeddings provide a more robust, flexible predictor of syntactic choices than verb-level statistics, illustrating the value of large corpora and neural representations for psycholinguistic modeling.

Abstract

Speakers often have multiple ways to express the same meaning. The Uniform Information Density (UID) hypothesis suggests that speakers exploit this variability to maintain a consistent rate of information transmission during language production. Building on prior work linking UID to syntactic reduction, we revisit the finding that the optional complementizer $\textit{that}$ in English complement clauses is more likely to be omitted when the clause has low information density (i.e., more predictable). We advance this line of research by analyzing a large-scale, contemporary conversational corpus and using machine learning and neural language models to refine estimates of information density. Our results replicated the established relationship between information density and $\textit{that}$-mentioning. However, we found that previous measures of information density based on matrix verbs' subcategorization probability capture substantial idiosyncratic lexical variation. By contrast, estimates derived from contextual word embeddings account for additional variance in patterns of complementizer usage.

Uniform Information Density and Syntactic Reduction: Revisiting $\textit{that}$-Mentioning in English Complement Clauses

TL;DR

This study revisits Uniform Information Density (UID) as a driver of syntactic reduction by examining that-mentioning in English complement clauses using the CANDOR corpus and neural language models. It compares information density estimates from verb subcategorization probabilities with contextual word embeddings derived from GPT-2 Small, finding that embedding-based density better accounts for that-mentioning after controlling for verb-specific effects. The embedding-based approach explains additional variance over traditional lexical predictors and remains predictive even when verb identity is modeled as a random effect, whereas verb-based density largely disappears under the same treatment. The results support UID in naturalistic speech but also show that high-level structural cues captured by embeddings provide a more robust, flexible predictor of syntactic choices than verb-level statistics, illustrating the value of large corpora and neural representations for psycholinguistic modeling.

Abstract

Speakers often have multiple ways to express the same meaning. The Uniform Information Density (UID) hypothesis suggests that speakers exploit this variability to maintain a consistent rate of information transmission during language production. Building on prior work linking UID to syntactic reduction, we revisit the finding that the optional complementizer in English complement clauses is more likely to be omitted when the clause has low information density (i.e., more predictable). We advance this line of research by analyzing a large-scale, contemporary conversational corpus and using machine learning and neural language models to refine estimates of information density. Our results replicated the established relationship between information density and -mentioning. However, we found that previous measures of information density based on matrix verbs' subcategorization probability capture substantial idiosyncratic lexical variation. By contrast, estimates derived from contextual word embeddings account for additional variance in patterns of complementizer usage.

Paper Structure

This paper contains 30 sections, 1 equation, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Psycholinguistic Evidence for UID
  4. Neural Language Model and Structural Knowledge
  5. Structural Predictability Model
  6. Linguistic Features
  7. Contextual Word Embeddings
  8. Training Data
  9. Model Architecture, Training, and Evaluation
  10. Structural Predictability Model Results
  11. Information Density and that-Mentioning
  12. Data
  13. Control variables
  14. Availability-Based Production
  15. Syntactic Priming
  16. ...and 15 more sections

Figures (4)

  • Figure 1: Illustration of per-word information density (note that the solid line represents a hypothetical channel capacity, and that the information density values are schematic only).
  • Figure 2: Effects of information density (by verb Subcategorization Probability) on that-mentioning. The red line is a logistic regression fit estimated as a binomial GLMM. Each dot represents a verb, while the shading reflects its frequency, with darker shades corresponding to more frequent verbs.
  • Figure 3: Effects of information density (by contextual word embeddings) on that-mentioning. The red line is a logistic regression fit estimated as a binomial GLMM.
  • Figure 4: Effects of information density (by verb Subcategorization Probability) on that-mentioning. The red line is a logistic regression fit estimated as a binomial GLMM. Each dot represents a verb.