Improving Interpretability of Lexical Semantic Change with Neurobiological Features

TL;DR

This work addresses the interpretability gap in lexical semantic change by mapping contextualized embeddings from BERT into a Binder neurobiological feature space of 65 interpretable dimensions. A regression model learns $\mathbf{b}_w = \psi(\mathbf{r}_w)$ to translate semantic representations into neurobiological features, enabling quantitative interpretation of meaning shifts and facilitating analyses with Sparse PCA. The approach yields competitive LSC detection performance, reveals new LSC types, and enables detection of amelioration and pejoration by focusing on targeted Binder features. The method enhances interpretability and offers a practical framework for analyzing historical semantic shifts with neurobiological grounding, while highlighting limitations and future directions such as broader LSC typologies and alternative representations.

Abstract

Lexical Semantic Change (LSC) is the phenomenon in which the meaning of a word change over time. Most studies on LSC focus on improving the performance of estimating the degree of LSC, however, it is often difficult to interpret how the meaning of a word change. Enhancing the interpretability of LSC is a significant challenge as it could lead to novel insights in this field. To tackle this challenge, we propose a method to map the semantic space of contextualized embeddings of words obtained by a pre-trained language model to a neurobiological feature space. In the neurobiological feature space, each dimension corresponds to a primitive feature of words, and its value represents the intensity of that feature. This enables humans to interpret LSC systematically. When employed for the estimation of the degree of LSC, our method demonstrates superior performance in comparison to the majority of the previous methods. In addition, given the high interpretability of the proposed method, several analyses on LSC are carried out. The results demonstrate that our method not only discovers interesting types of LSC that have been overlooked in previous studies but also effectively searches for words with specific types of LSC.

Figures (8)

• Figure 1: Binder feature values for "coffee" and "dog"
• Figure 2: LSC vectors for plane and terrific
• Figure 3: Eigenvectors obtained by Sparse PCA. The number of principal components is set to 10. The horizontal and vertical axes represent the 65 Binder features and the 10 principal components, respectively.
• Figure 4: Distributions of the usage types for bluegrass, explorer, and warming
• Figure 5: LSC scores for positive features