Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Partial Colexifications Improve Concept Embeddings

Arne Rubehn, Johann-Mattis List

TL;DR

The paper addresses the challenge of representing cross-linguistic concepts by leveraging colexification networks. It advances the field by incorporating partial colexifications (affix and overlap) alongside full colexifications and evaluating with three graph-embedding methods (SDNE, Node2Vec, ProNE) on three tasks: lexical semantic similarity, semantic change prediction, and word association prediction. Results demonstrate that partial colexifications improve concept embeddings across tasks, with affix colexifications providing the most consistent gains and full+affix embeddings delivering the strongest overall performance. This work enhances cross-linguistic meaning modeling and offers a resource and methodology that could benefit historical linguistics, typology, and multilingual NLP applications.

Abstract

While the embedding of words has revolutionized the field of Natural Language Processing, the embedding of concepts has received much less attention so far. A dense and meaningful representation of concepts, however, could prove useful for several tasks in computational linguistics, especially those involving cross-linguistic data or sparse data from low resource languages. First methods that have been proposed so far embed concepts from automatically constructed colexification networks. While these approaches depart from automatically inferred polysemies, attested across a larger number of languages, they are restricted to the word level, ignoring lexical relations that would only hold for parts of the words in a given language. Building on recently introduced methods for the inference of partial colexifications, we show how they can be used to improve concept embeddings in meaningful ways. The learned embeddings are evaluated against lexical similarity ratings, recorded instances of semantic shift, and word association data. We show that in all evaluation tasks, the inclusion of partial colexifications lead to improved concept representations and better results. Our results further show that the learned embeddings are able to capture and represent different semantic relationships between concepts.

Partial Colexifications Improve Concept Embeddings

TL;DR

The paper addresses the challenge of representing cross-linguistic concepts by leveraging colexification networks. It advances the field by incorporating partial colexifications (affix and overlap) alongside full colexifications and evaluating with three graph-embedding methods (SDNE, Node2Vec, ProNE) on three tasks: lexical semantic similarity, semantic change prediction, and word association prediction. Results demonstrate that partial colexifications improve concept embeddings across tasks, with affix colexifications providing the most consistent gains and full+affix embeddings delivering the strongest overall performance. This work enhances cross-linguistic meaning modeling and offers a resource and methodology that could benefit historical linguistics, typology, and multilingual NLP applications.

Abstract

While the embedding of words has revolutionized the field of Natural Language Processing, the embedding of concepts has received much less attention so far. A dense and meaningful representation of concepts, however, could prove useful for several tasks in computational linguistics, especially those involving cross-linguistic data or sparse data from low resource languages. First methods that have been proposed so far embed concepts from automatically constructed colexification networks. While these approaches depart from automatically inferred polysemies, attested across a larger number of languages, they are restricted to the word level, ignoring lexical relations that would only hold for parts of the words in a given language. Building on recently introduced methods for the inference of partial colexifications, we show how they can be used to improve concept embeddings in meaningful ways. The learned embeddings are evaluated against lexical similarity ratings, recorded instances of semantic shift, and word association data. We show that in all evaluation tasks, the inclusion of partial colexifications lead to improved concept representations and better results. Our results further show that the learned embeddings are able to capture and represent different semantic relationships between concepts.

Paper Structure

This paper contains 19 sections, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Word Embeddings
  4. Colexification Networks
  5. Graph Embeddings
  6. Concept Embeddings
  7. Materials and Methods
  8. Colexification Data
  9. Concept Embeddings
  10. Baselines
  11. Experiments
  12. Modeling Lexical Semantic Similarity
  13. Predicting Semantic Change
  14. Predicting Word Associations
  15. Implementation
  16. ...and 4 more sections

Figures (2)

  • Figure 1: Two-dimensional visualizations of embeddings only learned on full colexification data (left) and embeddings learned on full and affix colexifications (right), created using t-SNE, for a small list of concepts taken from the Swadesh list of 100 items Swadesh1955.
  • Figure 2: Two-dimensional visualization of embeddings for concepts from the Swadesh-100 list (Swadesh 1955) that are present in all three colexification networks, created using t-SNE.