What Can String Probability Tell Us About Grammaticality?

TL;DR

The paper investigates whether string probability meaningfully reflects grammatical knowledge by proposing a formal framework where $P(\boldsymbol{s})$ arises from underlying messages and grammatical realizations. It derives three predictions about how grammaticality, meaning, and probability relate, and tests them empirically on 280K English/Chinese sentence pairs using multiple datasets and models. The results support a correlation between grammatical and ungrammatical log-probabilities within meaning-matched minimal pairs and reveal alignment with human acceptability in English, but show weak separability between grammatical and ungrammatical strings overall, even after several probability-transformations. The work provides theoretical grounding for minimal-pair probability analyses and highlights the need to carefully control for meaning when evaluating grammatical knowledge in language models, with implications for cross-linguistic evaluation and future grammaticality metrics.

Abstract

What have language models (LMs) learned about grammar? This question remains hotly debated, with major ramifications for linguistic theory. However, since probability and grammaticality are distinct notions in linguistics, it is not obvious what string probabilities can reveal about an LM's underlying grammatical knowledge. We present a theoretical analysis of the relationship between grammar, meaning, and string probability, based on simple assumptions about the generative process of corpus data. Our framework makes three predictions, which we validate empirically using 280K sentence pairs in English and Chinese: (1) correlation between the probability of strings within minimal pairs, i.e., string pairs with minimal semantic differences; (2) correlation between models' and humans' deltas within minimal pairs; and (3) poor separation in probability space between unpaired grammatical and ungrammatical strings. Our analyses give theoretical grounding for using probability to learn about LMs' structural knowledge, and suggest directions for future work in LM grammatical evaluation.

Figures (8)

• Figure 1: (a) Illustration of messages (left) and strings (right) in toy domain. Blue = grammatical strings. Red = ungrammatical strings. (b) Surprisal (negative log probability) assigned to toy strings by GPT-2.
• Figure 2: (a) \ref{['pred:gram-ungram-corr']}a: Logprobs of paired grammatical ($x$-axis) and ungrammatical ($y$-axis) sentences are correlated. Dashed line: $x=y$. (b) \ref{['pred:gram-ungram-corr']}b: Correlation between grammatical and ungrammatical logprobs ($y$-axis) generally decreases as within-pair cosine distance ($x$-axis) increases.
• Figure 3: (a) \ref{['pred:model-human']}a: Deltas between grammatical and ungrammatical strings are correlated between models and humans. (b) \ref{['pred:model-human']}b: Correlation between model and human deltas ($y$-axis) generally decreases as within-pair cosine distance ($x$-axis) increases. Note this pattern is only apparent in the English data.
• Figure 4: Evaluation of \ref{['pred:no-separation']}. (a) Distributions of scores are highly overlapping across grammatical and ungrammatical sentences (pooled across datasets). (b) Poor separability (area under receiver operating characteristic curve, or AUC) achieved by each model and probability transformation. Horizontal line at 0.5 indicates no separation. For dataset-specific results, see \ref{['sec:appendix-no-separation']}, \ref{['fig:pooled-scores-all-datasets', 'fig:auc-all-datasets']}.
• Figure 5: Score distributions for grammatical and ungrammatical sentences from each English dataset.

Theorems & Definitions (4)

• Definition 1
• Definition 2
• Definition 3
• Definition 4