Translate-and-Revise: Boosting Large Language Models for Constrained Translation

TL;DR

This work tackles constrained translation by addressing a key limitation of LLMs: failing to follow predefined constraints. It introduces Translate-and-Revise (TAR), a two-stage framework that first translates with constraint-aware prompts and then uses an iterative reviser to fix unmet constraints, mitigating the memo trap. Across four constrained translation datasets with lexical and structural constraints, TAR achieves state-of-the-art constraint fidelity (CCR) and competitive translation quality, outperforming base LLMs and many data-augmentation NMT baselines. The method generalizes to multiple LLMs (e.g., GPT-3, GPT-4, Qwen) and to domain-agnostic NMT models, demonstrating robust improvements in constrained translation and highlighting the value of explicit feedback-driven revision.

Abstract

Imposing constraints on machine translation systems presents a challenging issue because these systems are not trained to make use of constraints in generating adequate, fluent translations. In this paper, we leverage the capabilities of large language models (LLMs) for constrained translation, given that LLMs can easily adapt to this task by taking translation instructions and constraints as prompts. However, LLMs cannot always guarantee the adequacy of translation, and, in some cases, ignore the given constraints. This is in part because LLMs might be overly confident in their predictions, overriding the influence of the constraints. To overcome this overiding behaviour, we propose to add a revision process that encourages LLMs to correct the outputs by prompting them about the constraints that have not yet been met. We evaluate our approach on four constrained translation tasks, encompassing both lexical and structural constraints in multiple constraint domains. Experiments show 15\% improvement in constraint-based translation accuracy over standard LLMs and the approach also significantly outperforms neural machine translation (NMT) state-of-the-art methods.

Figures (5)

• Figure 1: Given source language input $X$ and constraint pairs, a Translator produces an initial translation $Y_0$ where COVID-19 is translated as "新冠". Subsequently, a Reviser iteratively revise the translation $Y_i$ to a better one $Y_{i+1}$, correctly translating COVID-19 as "新型冠状病毒".
• Figure 2: Two stages of TAR, in the Translate stage, constraints $\langle S,T \rangle$ are incorporated into the prompt to enable the model to generate preliminary translation results that meet the constraints to a certain extent. In the Revise stage, LLMs revise the flawed translation results $Y^{flawed}$ with uncompleted constraints $\langle S,T \rangle^{un}$. The sections shaded in blue and yellow respectively represent the important parts of the two stages.
• Figure 3: TAR results on WMT21 TT using Qwen, ChatGPT, GPT-3 (text-davinci-003) and GPT-4. Here "w/o TAR" represents the use of the conventional translation prompt. "TAR w/o revision" indicates the use of a prompt with constraints, but without reviser. Meanwhile, "TAR" denotes the full method that includes revisions.
• Figure 4: (a) Improvements in CCR with each iteration. (b) Red denotes consistent template use across three iterations, while blue indicates alternating templates.
• Figure 5: (a) The impact of increasing the number of constraints on BLEU. (b) The effect of increasing the number of constraints on CCR.