Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Self-Augmented In-Context Learning for Unsupervised Word Translation

Yaoyiran Li, Anna Korhonen, Ivan Vulić

TL;DR

This work tackles unsupervised bilingual lexicon induction by leveraging large language models more effectively. It introduces Self-Augmented In-Context Learning (SAIL), which iteratively extracts high-confidence word translation pairs via zero-shot prompts and refines them through few-shot in-context learning, then uses the resulting seed lexicon to improve translation inference. Across XLING and PanLex-BLI benchmarks, SAIL achieves state-of-the-art unsupervised BLI performance and consistently outperforms established mapping-based baselines, with strong statistical significance. The approach highlights the potential of self-augmented in-context strategies for cross-lingual lexicon induction, while acknowledging limitations related to language coverage and computational requirements; code is publicly available for reproducibility and further study.

Abstract

Recent work has shown that, while large language models (LLMs) demonstrate strong word translation or bilingual lexicon induction (BLI) capabilities in few-shot setups, they still cannot match the performance of 'traditional' mapping-based approaches in the unsupervised scenario where no seed translation pairs are available, especially for lower-resource languages. To address this challenge with LLMs, we propose self-augmented in-context learning (SAIL) for unsupervised BLI: starting from a zero-shot prompt, SAIL iteratively induces a set of high-confidence word translation pairs for in-context learning (ICL) from an LLM, which it then reapplies to the same LLM in the ICL fashion. Our method shows substantial gains over zero-shot prompting of LLMs on two established BLI benchmarks spanning a wide range of language pairs, also outperforming mapping-based baselines across the board. In addition to achieving state-of-the-art unsupervised BLI performance, we also conduct comprehensive analyses on SAIL and discuss its limitations.

Self-Augmented In-Context Learning for Unsupervised Word Translation

TL;DR

This work tackles unsupervised bilingual lexicon induction by leveraging large language models more effectively. It introduces Self-Augmented In-Context Learning (SAIL), which iteratively extracts high-confidence word translation pairs via zero-shot prompts and refines them through few-shot in-context learning, then uses the resulting seed lexicon to improve translation inference. Across XLING and PanLex-BLI benchmarks, SAIL achieves state-of-the-art unsupervised BLI performance and consistently outperforms established mapping-based baselines, with strong statistical significance. The approach highlights the potential of self-augmented in-context strategies for cross-lingual lexicon induction, while acknowledging limitations related to language coverage and computational requirements; code is publicly available for reproducibility and further study.

Abstract

Recent work has shown that, while large language models (LLMs) demonstrate strong word translation or bilingual lexicon induction (BLI) capabilities in few-shot setups, they still cannot match the performance of 'traditional' mapping-based approaches in the unsupervised scenario where no seed translation pairs are available, especially for lower-resource languages. To address this challenge with LLMs, we propose self-augmented in-context learning (SAIL) for unsupervised BLI: starting from a zero-shot prompt, SAIL iteratively induces a set of high-confidence word translation pairs for in-context learning (ICL) from an LLM, which it then reapplies to the same LLM in the ICL fashion. Our method shows substantial gains over zero-shot prompting of LLMs on two established BLI benchmarks spanning a wide range of language pairs, also outperforming mapping-based baselines across the board. In addition to achieving state-of-the-art unsupervised BLI performance, we also conduct comprehensive analyses on SAIL and discuss its limitations.
Paper Structure (13 sections, 3 figures, 9 tables)

This paper contains 13 sections, 3 figures, 9 tables.

Table of Contents

  1. Introduction and Motivation
  2. Methodology
  3. Experimental Setup
  4. Results and Discussion
  5. Main Results
  6. Further Analyses
  7. Conclusion
  8. Languages
  9. Impact of with PanLex-BLI
  10. Templates
  11. Reproducibility Checklist
  12. Details of ChatGPT Experiments
  13. Full BLI Results

Figures (3)

  • Figure 1: Top-1 accuracy ($\times 100\%$) averaged over $20$ XLING BLI directions with respect to $N_{it}$. $N_{it}=0$ yields the Zero-Shot baseline.
  • Figure 2: Top-1 accuracy on a subset of XLING with respect to $N_{f}$. $N_{f}=0$ yields the Zero-Shot baseline.
  • Figure 3: Top-1 accuracy ($\times 100\%$) averaged over $6$ PanLex-BLI BLI directions with respect to $N_{it}$. $N_{it}=0$ yields the Zero-Shot baseline.