Semantics or spelling? Probing contextual word embeddings with orthographic noise

TL;DR

The paper questions the assumption that pretrained language model hidden states serve as reliable semantic proxies for contextual word embeddings (CWEs). By introducing minimal orthographic noise (a single character swap) in alphabetic words across authentic contexts and comparing CWEs with both edited and unedited forms, the authors quantify representational similarity using $cos(\bar{\mathbf{w}}, \bar{\mathbf{w}}_{edit})$ and $\rho_{\bar{\mathbf{w}}, \bar{\mathbf{w}}_{edit}}$ across diverse PLMs. They find that CWEs are highly sensitive to input perturbations, with sensitivity correlating to tokenization and the number of input tokens per word; context can mitigate this but token-length effects persist. These results challenge the notion that CWEs primarily reflect word-level semantics and urge caution when interpreting CWE similarity in semantic analyses, especially under noisy data conditions or semantic-shift tasks. The study highlights the fragility of representational similarity measures and the need for careful methodological design when using CWEs as semantic proxies.

Abstract

Pretrained language model (PLM) hidden states are frequently employed as contextual word embeddings (CWE): high-dimensional representations that encode semantic information given linguistic context. Across many areas of computational linguistics research, similarity between CWEs is interpreted as semantic similarity. However, it remains unclear exactly what information is encoded in PLM hidden states. We investigate this practice by probing PLM representations using minimal orthographic noise. We expect that if CWEs primarily encode semantic information, a single character swap in the input word will not drastically affect the resulting representation,given sufficient linguistic context. Surprisingly, we find that CWEs generated by popular PLMs are highly sensitive to noise in input data, and that this sensitivity is related to subword tokenization: the fewer tokens used to represent a word at input, the more sensitive its corresponding CWE. This suggests that CWEs capture information unrelated to word-level meaning and can be manipulated through trivial modifications of input data. We conclude that these PLM-derived CWEs may not be reliable semantic proxies, and that caution is warranted when interpreting representational similarity

Figures (4)

• Figure 1: Similarities between CWEs and edited-CWEs, using cosine (top) and Spearman (bottom) for individual words without contexts (left) and with contexts (right). The shaded area around each line represents the 99% confidence interval.
• Figure 2: Distribution of token lengths for all alphabetic words (left) and English words (right), tokenized without (top) and with noise (bottom). For most models, English words tend to be tokenized with fewer tokens (top right). Noise tends to increase the token length, with most noisy words requiring 3 or more tokens instead of 1 or 2.
• Figure 3: Standardized similarities (Spearman), grouped by model family. Token length is measured as the number of tokens used to encode the original, unedited word. We show similarities both for CWEs generated with context (dashed line) and without (solid line), and the shaded area around each line represents the 99% confidence interval.
• Figure 4: Cosine similarities between CWEs and edited-CWEs, grouped by model family. Formatting is otherwise identical to Figure \ref{['fig:appendix_standardized']}.