Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Compositional Translation: A Novel LLM-based Approach for Low-resource Machine Translation

Armel Zebaze, Benoît Sagot, Rachel Bawden

TL;DR

CompTra introduces a three-stage, LLM-based translation framework that decomposes a source sentence into short, overlapping phrases, translates each phrase with retrieved demonstrations, and then combines these self-generated phrase translations to produce a final translation. The approach targets low-resource languages by leveraging short-phrase translation and similarity-based demonstrations, demonstrating consistent gains over strong similarity-based few-shot baselines across FLORES 200, NTREX 128, and TICO-19, using multiple LLMs. Ablations show the importance of the decomposition strategy, the retriever, and the translation step, with NLLB+CompTra and paraphrase-based decompositions offering notable benefits in certain settings. Overall, CompTra offers a scalable, reasoning-based MT paradigm that improves cross-domain LR translation, enabling smaller LLMs to achieve competitive performance while reducing reliance on extensive outside knowledge during inference.

Abstract

The ability of generative large language models (LLMs) to perform in-context learning has given rise to a large body of research into how best to prompt models for various natural language processing tasks. Machine Translation (MT) has been shown to benefit from in-context examples, in particular when they are semantically similar to the sentence to translate. In this paper, we propose a new LLM-based translation paradigm, compositional translation, to replace naive few-shot MT with similarity-based demonstrations. An LLM is used to decompose a sentence into simpler phrases, and then to translate each phrase with the help of retrieved demonstrations. Finally, the LLM is prompted to translate the initial sentence with the help of the self-generated phrase-translation pairs. Our intuition is that this approach should improve translation because these shorter phrases should be intrinsically easier to translate and easier to match with relevant examples. This is especially beneficial in low-resource scenarios, and more generally whenever the selection pool is small or out of domain. We show that compositional translation boosts LLM translation performance on a wide range of popular MT benchmarks, including FLORES 200, NTREX 128 and TICO-19. Code and outputs are available at https://github.com/ArmelRandy/compositional-translation

Compositional Translation: A Novel LLM-based Approach for Low-resource Machine Translation

TL;DR

CompTra introduces a three-stage, LLM-based translation framework that decomposes a source sentence into short, overlapping phrases, translates each phrase with retrieved demonstrations, and then combines these self-generated phrase translations to produce a final translation. The approach targets low-resource languages by leveraging short-phrase translation and similarity-based demonstrations, demonstrating consistent gains over strong similarity-based few-shot baselines across FLORES 200, NTREX 128, and TICO-19, using multiple LLMs. Ablations show the importance of the decomposition strategy, the retriever, and the translation step, with NLLB+CompTra and paraphrase-based decompositions offering notable benefits in certain settings. Overall, CompTra offers a scalable, reasoning-based MT paradigm that improves cross-domain LR translation, enabling smaller LLMs to achieve competitive performance while reducing reliance on extensive outside knowledge during inference.

Abstract

The ability of generative large language models (LLMs) to perform in-context learning has given rise to a large body of research into how best to prompt models for various natural language processing tasks. Machine Translation (MT) has been shown to benefit from in-context examples, in particular when they are semantically similar to the sentence to translate. In this paper, we propose a new LLM-based translation paradigm, compositional translation, to replace naive few-shot MT with similarity-based demonstrations. An LLM is used to decompose a sentence into simpler phrases, and then to translate each phrase with the help of retrieved demonstrations. Finally, the LLM is prompted to translate the initial sentence with the help of the self-generated phrase-translation pairs. Our intuition is that this approach should improve translation because these shorter phrases should be intrinsically easier to translate and easier to match with relevant examples. This is especially beneficial in low-resource scenarios, and more generally whenever the selection pool is small or out of domain. We show that compositional translation boosts LLM translation performance on a wide range of popular MT benchmarks, including FLORES 200, NTREX 128 and TICO-19. Code and outputs are available at https://github.com/ArmelRandy/compositional-translation

Paper Structure

This paper contains 50 sections, 2 figures, 23 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Using LLMs for Machine Translation
  4. Prompting and Compositionality
  5. Prompting LLMs for Machine Translation
  6. LLMs and Low-Resource Languages
  7. Methodology
  8. Experiments
  9. Experimental setup
  10. Datasets.
  11. Models.
  12. Evaluation Metrics.
  13. Baselines.
  14. Experimental Details.
  15. Results
  16. ...and 35 more sections

Figures (2)

  • Figure 1: An overview of Compositional Translation (CompTra). Given a sentence, our method prompts the LLM to decompose it into several phrases that use its words. For each phrase, we retrieve relevant in-context demonstrations (here four) through similarity search and use them to translate it in a few-shot setup. The phrase-translation couples obtained are then cleaned and provided to the LLM to help it translate the main sentence.
  • Figure 2: Impact of the number of in-context examples per phrase.