Developments in Sheaf-Theoretic Models of Natural Language Ambiguities

TL;DR

This work investigates how sheaf-theoretic models can capture natural-language ambiguities by extending contextuality analysis from lexical to discourse-level phenomena. It combines presheaf/theory-based representations with Contextuality-by-Default (CbD) to quantify contextuality via metrics such as CF and signalling considerations, applying them to basic anaphora and a generalized Winograd Schema. The authors report a high incidence of contextuality in a basic anaphora dataset (CF-enabled: 82.87%), and demonstrate contextuality in a generalized Winograd Schema through human judgments that produce a Bell-CHSH violation of $0.192 \pm 0.176$ (bootstrap significance $87\%$). These results establish a framework for analyzing discourse-level ambiguities with rigorous contextuality measures and point to future work using naturalistic data and large-language models to refine and deploy these models in NLP contexts.

Abstract

Sheaves are mathematical objects consisting of a base which constitutes a topological space and the data associated with each open set thereof, e.g. continuous functions defined on the open sets. Sheaves have originally been used in algebraic topology and logic. Recently, they have also modelled events such as physical experiments and natural language disambiguation processes. We extend the latter models from lexical ambiguities to discourse ambiguities arising from anaphora. To begin, we calculated a new measure of contextuality for a dataset of basic anaphoric discourses, resulting in a higher proportion of contextual models-82.9%-compared to previous work which only yielded 3.17% contextual models. Then, we show how an extension of the natural language processing challenge, known as the Winograd Schema, which involves anaphoric ambiguities can be modelled on the Bell-CHSH scenario with a contextual fraction of 0.096.

Figures (5)

• Figure 1: The geometric representation of the Bell-CHSH scenario and an empirical model for it.
• Figure 2: The left box shows the general PR prism schema of basic anaphoric ambiguities. The two outcomes $O_1$ and $O_2$ are two nouns which are the referents of the anaphor. The three observables $X_1$, $X_2$, and $X_3$ are the modifiers of the nouns. The right box shows an instance of the schema with adjective modifiers.
• Figure 3: A screenshot of the template of the questionnaire used to collect human judgments on the generalised Winograd Schema.
• Figure 4: Distributions of 11,052 examples of basic anaphoric ambiguities. The top histogram shows the distribution of the signalling fractions. We observed that 350 examples (3.17%) have a signalling fraction less than $1/6$, which is the threshold for conclusive contextuality according to the criterion of Emeriau2022. The bottom histogram shows the distribution of the Direct Influence of the CbD framework. We observed that 9159 examples (82.87%) have a Direct Influence of less than $2$, which is the threshold for contextuality in the CbD framework.
• Figure 5: A histogram of violation of Bell-CHSH inequality for 100,000 bootstrap samples.

Theorems & Definitions (2)

• Definition 1: Winograd Schema scenario
• Definition 2: Generalised Winograd Schema scenario