Adaptive Few-shot Prompting for Machine Translation with Pre-trained Language Models

TL;DR

This paper tackles prompt sensitivity in LLM-based machine translation by introducing Adaptive Few-shot Prompting (AFSP), which automatically selects suitable translation demonstrations using a hybrid, LLM-driven embedding retrieval and then reranks multiple LLM outputs with a self-supervised re-ranker. AFSP comprises three core components: a fixed prompt template filled with top-k semantically similar demonstrations retrieved via dense, sparse, and multi-vector embeddings from the deployed LLM, generation of multiple output candidates, and a reranker trained with negative sampling to ensure semantic consistency between source and target. A key methodological contribution is the hybrid demonstration retrieval score $s_{rank} = \alpha_1 s_{dense} + \alpha_2 s_{sparse} + \alpha_3 s_{multi}$, enabling robust demonstration selection without external embedding models. The authors also contribute a high-quality Diplomatic Chinese-English parallel dataset (5,528 sentences) and validate AFSP on this and the UN Chinese-English subset, reporting consistent improvements across BLEU, METEOR, ROUGE, CHRF, and COMET-Kiwi, as well as favorable human judgments. Overall, AFSP advances practical MT with LLMs by automatically tailoring demonstrations and leveraging self-supervised reranking, with significant implications for deploying prompt-based MT in dynamic, real-world settings.

Abstract

Recently, Large language models (LLMs) with in-context learning have demonstrated remarkable potential in handling neural machine translation. However, existing evidence shows that LLMs are prompt-sensitive and it is sub-optimal to apply the fixed prompt to any input for downstream machine translation tasks. To address this issue, we propose an adaptive few-shot prompting (AFSP) framework to automatically select suitable translation demonstrations for various source input sentences to further elicit the translation capability of an LLM for better machine translation. First, we build a translation demonstration retrieval module based on LLM's embedding to retrieve top-k semantic-similar translation demonstrations from aligned parallel translation corpus. Rather than using other embedding models for semantic demonstration retrieval, we build a hybrid demonstration retrieval module based on the embedding layer of the deployed LLM to build better input representation for retrieving more semantic-related translation demonstrations. Then, to ensure better semantic consistency between source inputs and target outputs, we force the deployed LLM itself to generate multiple output candidates in the target language with the help of translation demonstrations and rerank these candidates. Besides, to better evaluate the effectiveness of our AFSP framework on the latest language and extend the research boundary of neural machine translation, we construct a high-quality diplomatic Chinese-English parallel dataset that consists of 5,528 parallel Chinese-English sentences. Finally, extensive experiments on the proposed diplomatic Chinese-English parallel dataset and the United Nations Parallel Corpus (Chinese-English part) show the effectiveness and superiority of our proposed AFSP.

Figures (2)

• Figure 1: Illustration of translation results from different prompts using Llama3-8B. Our adaptive prompting framework can adaptively select the suitable prompt for an input text, yielding a better translation.
• Figure 2: The overview of our proposed Adaptive Few-shot Prompting (AFSP) framework.