Compensating for Data with Reasoning: Low-Resource Machine Translation with LLMs

TL;DR

This work tackles translation for extremely low-resource languages by leveraging in-context learning with Fragment-Shot prompting, which selects translation examples to maximize syntactic coverage. It introduces Pivoted Fragment-Shot to enable translations between language variants without direct parallel data, using a pivot such as Italian. Experiments across multiple LLMs show that syntactic coverage and strong reasoning capabilities significantly improve translations between Ladin variants and Italian, while gains are limited for high-resource directions; NMT remains superior for translations into LR languages. The study provides public code and retrieval corpora, highlighting the potential and limits of prompting-based approaches in low-resource MT and outlining directions for future research and scalability.

Abstract

Large Language Models (LLMs) have demonstrated strong capabilities in multilingual machine translation, sometimes even outperforming traditional neural systems. However, previous research has highlighted the challenges of using LLMs, particularly with prompt engineering, for low-resource languages. In this work, we introduce Fragment-Shot Prompting, a novel in-context learning method that segments input and retrieves translation examples based on syntactic coverage, along with Pivoted Fragment-Shot, an extension that enables translation without direct parallel data. We evaluate these methods using GPT-3.5, GPT-4o, o1-mini, LLaMA-3.3, and DeepSeek-R1 for translation between Italian and two Ladin variants, revealing three key findings: (1) Fragment-Shot Prompting is effective for translating into and between the studied low-resource languages, with syntactic coverage positively correlating with translation quality; (2) Models with stronger reasoning abilities make more effective use of retrieved knowledge, generally produce better translations, and enable Pivoted Fragment-Shot to significantly improve translation quality between the Ladin variants; and (3) prompt engineering offers limited, if any, improvements when translating from a low-resource to a high-resource language, where zero-shot prompting already yields satisfactory results. We publicly release our code and the retrieval corpora.

Figures (2)

• Figure 1: Fragment-Shot Prompting
• Figure 2: Example of Pivoted-Fragments Prompting and the corresponding reasoning employed by different LLMs.