Graph-based Clustering for Detecting Semantic Change Across Time and Languages

TL;DR

The paper tackles semantic change detection across time and languages by introducing a graph-based clustering framework that leverages contextualized embeddings. It combines a semantic-tree representation of word senses with temporal dynamics and an explicit cross-lingual (spatial) graph, using neighbor-based distance and bipartite matching to track sense evolution. Empirically, the method substantially improves binary semantic-change detection on SemEval2020 across English, German, Latin, and Swedish and provides a powerful visualization tool for exploring intra- and inter-language changes, with ablation analyses underscoring the value of time-dependent clustering and the neighbor-based distance. Limitations include reduced ranking performance for non-English languages and the ongoing challenge of cross-language embedding alignment, pointing to future work on multilingual embeddings and richer benchmarks. Overall, the approach offers a versatile, interpretable framework for diachronic and cross-linguistic semantic analysis with potential applications in historical linguistics and multilingual NLP.

Abstract

Despite the predominance of contextualized embeddings in NLP, approaches to detect semantic change relying on these embeddings and clustering methods underperform simpler counterparts based on static word embeddings. This stems from the poor quality of the clustering methods to produce sense clusters -- which struggle to capture word senses, especially those with low frequency. This issue hinders the next step in examining how changes in word senses in one language influence another. To address this issue, we propose a graph-based clustering approach to capture nuanced changes in both high- and low-frequency word senses across time and languages, including the acquisition and loss of these senses over time. Our experimental results show that our approach substantially surpasses previous approaches in the SemEval2020 binary classification task across four languages. Moreover, we showcase the ability of our approach as a versatile visualization tool to detect semantic changes in both intra-language and inter-language setups. We make our code and data publicly available.

Figures (12)

• Figure 1: Representation of the semantic changes for 'mouse' in our temporal dynamic graph. Blue nodes indicate the acquisition of a new meaning over time, while black nodes indicate unchanged word meanings.
• Figure 2: (Top) Representation of polysemous meanings of a word $w$ in a semantic-tree graph. (Bottom) Graph representation of 'mouse' as the root node, generated by applying our approach to the English Wikipedia corpus.
• Figure 3: Representation of semantic changes over time in a temporal dynamic graph.
• Figure 4: Representation of polysemous meanings of a mutual word translation pair in a spatial dynamic graph.
• Figure 5: Comparison of sense clusters for the word 'bit' between the time-dependent (at time $t-1$ and $t$) and time-independent setups. Color indicates the cluster assignment of each point. A dot point represents the high-frequency word sense (a small piece), while a '+' indicates the low-frequency sense (binary digit).