PWESuite: Phonetic Word Embeddings and Tasks They Facilitate

TL;DR

Three methods that use articulatory features to build phonetically informed word embeddings are developed that address the inconsistent evaluation of existing phonetic word embedding methods and contribute a task suite to fairly evaluate past, current, and future methods.

Abstract

Mapping words into a fixed-dimensional vector space is the backbone of modern NLP. While most word embedding methods successfully encode semantic information, they overlook phonetic information that is crucial for many tasks. We develop three methods that use articulatory features to build phonetically informed word embeddings. To address the inconsistent evaluation of existing phonetic word embedding methods, we also contribute a task suite to fairly evaluate past, current, and future methods. We evaluate both (1) intrinsic aspects of phonetic word embeddings, such as word retrieval and correlation with sound similarity, and (2) extrinsic performance on tasks such as rhyme and cognate detection and sound analogies. We hope our task suite will promote reproducibility and inspire future phonetic embedding research.

Figures (5)

• Figure 1: Embedding function $f$ projects words in various forms (left) to a vector space (right) such that words with a similar pronunciation (e.g., ocean and motion) are closer than words with a dissimilar pronunciation (e.g., ocean and soybean).
• Figure 2: Spearman (upper left) and Pearson (lower right) correlations between performance on suite tasks. All models from \ref{['tab:evaluation_all']} are used.
• Figure 3: Suite score of Metric Learner with articulatory features trained on one language and evaluated on another one. Diagonal shows models trained and evaluated on the same language.
• Figure 4: T-SNE projection of articulatory distance and embedding spaces from the metric learning models with articulatory or character features. Each point corresponds to one English word. Differently coloured clusters were selected in the articulatory distance space (left) and highlighted in other spaces. $d$ is the average distance within the clusters normalized with average distance between points (unitless). Articulatory Features (center) result in tighter clusters than Characters (right).
• Figure 5: Metric Learner performance with varying dimensionality (top) and varying training data size (bottom) with articulatory features. Bands show 95% confidence intervals from t-distribution.