LLM-based Translation Inference with Iterative Bilingual Understanding

TL;DR

IBUT addresses Understanding Distortion in LLM-based MT by generating bilingual contextual understanding and leveraging dual-learning signals to iteratively refine this understanding. It consists of four components—Understanding Generation, Alignment Judgment, Iterative Refinement, and Understanding-Based Translation—utilizing cross-lingual capabilities to improve translation accuracy. Across WMT22/23, Commonsense MT, and Cultural MT, IBUT outperforms strong baselines (e.g., ChatGPT, GPT-4, MAD, MAPS) on COMET, BLEURT, and BLEU, with supportive human evaluations. While achieving robust cross-domain performance and model-generalizability, IBUT incurs higher computational costs due to its iterative, multi-step process, marking a trade-off between translation quality and resource usage.

Abstract

The remarkable understanding and generation capabilities of large language models (LLMs) have greatly improved translation performance. However, incorrect understanding of the sentence to be translated can degrade translation quality. To address this issue, we proposed a novel Iterative Bilingual Understanding Translation (IBUT) method based on the cross-lingual capabilities of LLMs and the dual characteristics of translation tasks. The cross-lingual capability of LLMs enables the generation of contextual understanding for both the source and target languages separately. Furthermore, the dual characteristics allow IBUT to generate effective cross-lingual feedback, iteratively refining contextual understanding, thereby reducing errors and improving translation performance. Experimental results showed that the proposed IBUT outperforms several strong comparison methods, especially being generalized to multiple domains (e.g., news, commonsense, and cultural translation benchmarks).

Figures (6)

• Figure 1: Illustration of the LLMs translation paradigm based on contextual understanding (Fig a). A commonsense domain example of LLM (gpt-3.5-turbo) translation from Chinese to English (Fig b).
• Figure 2: IBUT translation framework. The process involves first generating a bilingual understanding of the translation input sentence using an LLM. Next, verbal feedback is obtained via LLM, informed by the translation input and the bilingual understanding. This feedback is then used to further refine the bilingual understanding. The final step involves using LLM to perform the translation, leveraging both the bilingual understanding and the original input sentence. Gray text indicates English annotations for the Chinese.
• Figure 3: The experiment measures the relationship between the improvement in contextual understanding quality and translation performance during iterative refinement.
• Figure 4: Analysis of the experimental setup for assessing the impact of the Iterative Refinement part on translation performance.
• Figure 5: Human preference study comparing ChatGPT, Refine, MAPS, and MAD.