Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Speakers Fill Lexical Semantic Gaps with Context

Tiago Pimentel, Rowan Hall Maudslay, Damián Blasi, Ryan Cotterell

TL;DR

This work formalizes lexical ambiguity and contextual uncertainty within an information-theoretic framework, defining lexical ambiguity as $\mathrm{H}(\mathrm{M}\mid \mathrm{W})$ and contextual uncertainty as $\mathrm{H}(\mathrm{W}\mid \mathrm{C})$. It proposes two estimation strategies for lexical ambiguity: a WordNet-based discrete-senses approach and a continuous-space approach using multilingual BERT embeddings, with entropy bounds derived for both. Through analyses across 6 high-resource languages (WordNet) and 18 typologically diverse languages (BERT), the authors find robust, significant negative correlations between lexical ambiguity and contextual uncertainty, supporting the hypothesis that speakers compensate for ambiguity by making contexts more informative. The results suggest a broad, cross-linguistic principle: efficient communication balances economy and clarity by aligning lexical ambiguity with contextual predictability, leveraging context to disambiguate ambiguous forms. These findings have implications for understanding language design and for cross-linguistic NLP work that relies on lexical meaning representation and contextual cues.

Abstract

Lexical ambiguity is widespread in language, allowing for the reuse of economical word forms and therefore making language more efficient. If ambiguous words cannot be disambiguated from context, however, this gain in efficiency might make language less clear -- resulting in frequent miscommunication. For a language to be clear and efficiently encoded, we posit that the lexical ambiguity of a word type should correlate with how much information context provides about it, on average. To investigate whether this is the case, we operationalise the lexical ambiguity of a word as the entropy of meanings it can take, and provide two ways to estimate this -- one which requires human annotation (using WordNet), and one which does not (using BERT), making it readily applicable to a large number of languages. We validate these measures by showing that, on six high-resource languages, there are significant Pearson correlations between our BERT-based estimate of ambiguity and the number of synonyms a word has in WordNet (e.g. $ρ= 0.40$ in English). We then test our main hypothesis -- that a word's lexical ambiguity should negatively correlate with its contextual uncertainty -- and find significant correlations on all 18 typologically diverse languages we analyse. This suggests that, in the presence of ambiguity, speakers compensate by making contexts more informative.

Speakers Fill Lexical Semantic Gaps with Context

TL;DR

This work formalizes lexical ambiguity and contextual uncertainty within an information-theoretic framework, defining lexical ambiguity as and contextual uncertainty as . It proposes two estimation strategies for lexical ambiguity: a WordNet-based discrete-senses approach and a continuous-space approach using multilingual BERT embeddings, with entropy bounds derived for both. Through analyses across 6 high-resource languages (WordNet) and 18 typologically diverse languages (BERT), the authors find robust, significant negative correlations between lexical ambiguity and contextual uncertainty, supporting the hypothesis that speakers compensate for ambiguity by making contexts more informative. The results suggest a broad, cross-linguistic principle: efficient communication balances economy and clarity by aligning lexical ambiguity with contextual predictability, leveraging context to disambiguate ambiguous forms. These findings have implications for understanding language design and for cross-linguistic NLP work that relies on lexical meaning representation and contextual cues.

Abstract

Lexical ambiguity is widespread in language, allowing for the reuse of economical word forms and therefore making language more efficient. If ambiguous words cannot be disambiguated from context, however, this gain in efficiency might make language less clear -- resulting in frequent miscommunication. For a language to be clear and efficiently encoded, we posit that the lexical ambiguity of a word type should correlate with how much information context provides about it, on average. To investigate whether this is the case, we operationalise the lexical ambiguity of a word as the entropy of meanings it can take, and provide two ways to estimate this -- one which requires human annotation (using WordNet), and one which does not (using BERT), making it readily applicable to a large number of languages. We validate these measures by showing that, on six high-resource languages, there are significant Pearson correlations between our BERT-based estimate of ambiguity and the number of synonyms a word has in WordNet (e.g. in English). We then test our main hypothesis -- that a word's lexical ambiguity should negatively correlate with its contextual uncertainty -- and find significant correlations on all 18 typologically diverse languages we analyse. This suggests that, in the presence of ambiguity, speakers compensate by making contexts more informative.

Paper Structure

This paper contains 20 sections, 20 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Ambiguity in Language
  3. Lexical Ambiguity
  4. Accounts of Lexical Ambiguity
  5. Ambiguity and Uncertainty
  6. Lexical Ambiguity
  7. Contextual Uncertainty
  8. Hypothesis: Why Should Ambiguity Correlate with Uncertainty?
  9. Computation and Approximation
  10. Lexical Ambiguity
  11. Discrete senses
  12. Continuous meaning space
  13. Contextual Uncertainty
  14. Data
  15. Discussion: WordNet vs. $\bm{\mathrm{BERT}{}}$-based approximations
  16. ...and 5 more sections

Figures (3)

  • Figure 1: The relationship between contextual uncertainty---how uncertain a word is given its context---and lexical ambiguity, across a diverse set of languages.
  • Figure 2: Correlating our $\mathrm{BERT}$-based estimate of lexical ambiguity with the number of senses in WordNet
  • Figure 3: Contextual uncertainty versus lexical ambiguity in a selection of languages. Each plot contains the scatter points (representing each word type), a robust linear regression and kernel density estimate regions. (From left to right; Top) WordNet: Arabic, English, Indonesian; (Bottom) $\mathrm{BERT}$: Arabic, English, Malayalam, Tagalog.