Words that make SENSE: Sensorimotor Norms in Learned Lexical Token Representations

TL;DR

This work addresses whether distributional word representations encode sensorimotor grounding by predicting the Lancaster sensorimotor norms from lexical embeddings. It introduces SENSE, a projection from embeddings to an 11-dimensional sensorimotor space, and validates it with Word2Vec, GloVe, and BERT CLS, plus a behavioral study of 281 participants using nonce words to test generalization. Findings show significant correlations between SENSE predictions and human judgments across 6 modalities, and sublexical analysis reveals a robust interoceptive phonostheme signal, suggesting that form-meaning associations can emerge from text-only data. These results demonstrate grounding in distributional models and point to future work on cross-linguistic sensorimotor encoding and phonological pattern discovery.

Abstract

While word embeddings derive meaning from co-occurrence patterns, human language understanding is grounded in sensory and motor experience. We present $\text{SENSE}$ $(\textbf{S}\text{ensorimotor }$ $\textbf{E}\text{mbedding }$ $\textbf{N}\text{orm }$ $\textbf{S}\text{coring }$ $\textbf{E}\text{ngine})$, a learned projection model that predicts Lancaster sensorimotor norms from word lexical embeddings. We also conducted a behavioral study where 281 participants selected which among candidate nonce words evoked specific sensorimotor associations, finding statistically significant correlations between human selection rates and $\text{SENSE}$ ratings across 6 of the 11 modalities. Sublexical analysis of these nonce words selection rates revealed systematic phonosthemic patterns for the interoceptive norm, suggesting a path towards computationally proposing candidate phonosthemes from text data.

Figures (13)

• Figure 1: Mean squared error (MSE) for the six perceptual modalities comparing Sense against the baseline model predicting mean training sensorimotor vector ${\bar{\mathbf{s}}}$ for all inputs. Error bars represent standard error of the MSE. The neural network substantially outperforms the baseline across all modalities, with lowest errors for gustatory and olfactory and highest for visual and auditory.
• Figure 2: Correlation between the rate of human selection vs SENSE rating for nonce words shown to the humans under the Interoceptive category ($r=0.73$).
• Figure 3: Sample words from the Lancaster Sensorimotor Norms dataset showing ratings across 11 dimensions (6 perceptual modalities and 5 action effectors).
• Figure 4: Correlation between human selection rate and SENSE ratings for nonce words in the auditory modality ($r = 0.69, p < .001$).
• Figure 5: Correlation between human selection rate and SENSE ratings for nonce words in the visual modality ($r = 0.56, p = .002$).