Word-Centered Semantic Graphs for Interpretable Diachronic Sense Tracking

TL;DR

The paper presents word-centered semantic graphs that integrate static distributional similarity with contextual substitutability to model diachronic semantic shift in long-span corpora. Sense structure is discovered via peripheral connectivity within word neighborhoods, and evolution is tracked through time-based cluster alignment and normalized cluster mass. Applied to NYT Magazine data (1980–2017) for trump, god, and post, the approach reveals event-driven sense reconfiguration for trump, semantic stability for god, and gradual broadening for post, all without predefined sense inventories. This framework yields an interpretable, compact representation of sense evolution suitable for historical linguistics and long-range language technologies.

Abstract

We propose an interpretable, graph-based framework for analyzing semantic shift in diachronic corpora. For each target word and time slice, we induce a word-centered semantic network that integrates distributional similarity from diachronic Skip-gram embeddings with lexical substitutability from time-specific masked language models. We identify sense-related structure by clustering the peripheral graph, align clusters across time via node overlap, and track change through cluster composition and normalized cluster mass. In an application study on a corpus of New York Times Magazine articles (1980 - 2017), we show that graph connectivity reflects polysemy dynamics and that the induced communities capture contrasting trajectories: event-driven sense replacement (trump), semantic stability with cluster over-segmentation effects (god), and gradual association shifts tied to digital communication (post). Overall, word-centered semantic graphs offer a compact and transparent representation for exploring sense evolution without relying on predefined sense inventories.

Figures (16)

• Figure 1: Word-centered semantic networks of the word trump in 2017. (Top) Edge: blue edges denote distributional similarity (Word2Vec), yellow edges denote substitution-based similarity (RoBERTa); if both relations hold, the edge is shown in yellow. (Bottom) Edge: cosine similarity between contextual (RoBERTa) embeddings of the connected nodes. Darker edges denote lower similarity. The network reflects a politically grounded semantic neighborhood dominated by institutional and partisan associations.
• Figure 2: Word-centered semantic networks of trump in 1980 (left) and 1990 (right). Blue edges indicate distributional similarity; yellow edges indicate substitution-based similarity. The 1980 network forms a dense, cohesive structure corresponding to the literal card-game sense (low polysemy), whereas the 1990 network exhibits disconnected communities associated with business-related meanings, indicating increased polysemy.
• Figure 3: Graph properties of the trump neighborhood over time. Blue line shows the number of nodes; orange line shows the number of edges. The peak around 2000 corresponds to a transitional polysemous phase, followed by a decline reflecting sense contraction and replacement.
• Figure 4: Sense usage distributions for trump derived from normalized cluster mass over time. (Top) Clusters aligned to the immediately preceding period, highlighting short-lived (event-driven) senses but yielding more fragmentation and a larger residual cluster. (Bottom) Clusters aligned across all historical periods, producing fewer, more stable senses while potentially absorbing newly emerging meanings into earlier clusters. Legend labels are assigned by manual inspection of cluster nodes.
• Figure 5: Relative frequency of target words trump (Blue), god (Orange), and post (Green) across nine time periods (1980-2017) in the New York Times Magazine corpus. The targets exhibit distinct frequency trajectories, supporting their use as contrasting probes for semantic change analysis.