Semi-Supervised Learning for Bilingual Lexicon Induction

TL;DR

The paper tackles bilingual lexicon induction without parallel data by semi-supervised alignment of monolingual word embeddings into a shared space using a Wasserstein-Procrustes framework and learns a ranking-based lexicon induction via Learning to Rank. It introduces RUBI, which leverages learning from a set of auxiliary languages via an A-C dictionary to improve A-B translation, and it incorporates a differentiable NDCG loss (ApproxNDCG) within the ranking model. Empirical results on $EN\rightarrow ES$ and over 20 languages show consistent state-of-the-art gains, e.g., $95.3\%$ accuracy for $EN\leftrightarrow ES$ and substantial improvements for distant pairs like $EN\rightarrow RU$. The work supports the hypothesis that cross-lingual embedding spaces share similar geometry and demonstrates the value of semi-supervised, multi-language information for BLI, offering a scalable route to high-quality dictionaries without direct parallel data.

Abstract

We consider the problem of aligning two sets of continuous word representations, corresponding to languages, to a common space in order to infer a bilingual lexicon. It was recently shown that it is possible to infer such lexicon, without using any parallel data, by aligning word embeddings trained on monolingual data. Such line of work is called unsupervised bilingual induction. By wondering whether it was possible to gain experience in the progressive learning of several languages, we asked ourselves to what extent we could integrate the knowledge of a given set of languages when learning a new one, without having parallel data for the latter. In other words, while keeping the core problem of unsupervised learning in the latest step, we allowed the access to other corpora of idioms, hence the name semi-supervised. This led us to propose a novel formulation, considering the lexicon induction as a ranking problem for which we used recent tools of this machine learning field. Our experiments on standard benchmarks, inferring dictionary from English to more than 20 languages, show that our approach consistently outperforms existing state of the art benchmark. In addition, we deduce from this new scenario several relevant conclusions allowing a better understanding of the alignment phenomenon.

Figures (7)

• Figure 1: Word embeddings alignment (in dimension 2).
• Figure 2: Loss function Impact.
• Figure 3: Group Size Impact.
• Figure 4: CSLS feature Impact.
• Figure 5: Query Size Impact.