Improving LLM-based Document-level Machine Translation with Multi-Knowledge Fusion

TL;DR

The paper addresses the challenge of document-level machine translation (DMT) with LLMs by moving beyond single-source inter-sentence context to a multi-knowledge fusion framework. It introduces a three-stage pipeline: (1) Document-Level Knowledge Acquisition to extract Summarization 𝒀^s and Entity Translation 𝒀^e from the source, (2) Single-Knowledge Integration to generate translations 𝒀^s, 𝒀^e, and a baseline 𝒀^b, and (3) Multi-Knowledge Fusion to select per-sentence best translations via $Y_i^f = \mathop{\mathrm{arg\,max}}\limits S(Y, X_i)$, forming 𝒀^f. The approach yields consistent improvements in COMET scores across eight translation directions and three LLMs, with average gains of 0.8, 0.6, and 0.4 for LLaMA3-8B-Instruct, Mistral-Nemo-Instruct, and GPT-4o-mini, respectively, and shows stronger benefits for English-centric directions. Analyses reveal that combining knowledge sources mitigates sentence-level variability, enhances coherence and lexical consistency (LTCR), and that summarization knowledge tends to contribute more than entity-translation knowledge. The work demonstrates that multi-knowledge fusion is a practical, resource-efficient enhancement for LLM-based DMT and suggests avenues for broader language coverage and dataset diversity.

Abstract

Recent studies in prompting large language model (LLM) for document-level machine translation (DMT) primarily focus on the inter-sentence context by flatting the source document into a long sequence. This approach relies solely on the sequence of sentences within the document. However, the complexity of document-level sequences is greater than that of shorter sentence-level sequences, which may limit LLM's ability in DMT when only this single-source knowledge is used. In this paper, we propose an enhanced approach by incorporating multiple sources of knowledge, including both the document summarization and entity translation, to enhance the performance of LLM-based DMT. Given a source document, we first obtain its summarization and translation of entities via LLM as the additional knowledge. We then utilize LLMs to generate two translations of the source document by fusing these two single knowledge sources, respectively. Finally, recognizing that different sources of knowledge may aid or hinder the translation of different sentences, we refine and rank the translations by leveraging a multi-knowledge fusion strategy to ensure the best results. Experimental results in eight document-level translation tasks show that our approach achieves an average improvement of 0.8, 0.6, and 0.4 COMET scores over the baseline without extra knowledge for LLaMA3-8B-Instruct, Mistral-Nemo-Instruct, and GPT-4o-mini, respectively.

Figures (6)

• Figure 1: Illustration of a professional translator translating a document from Chinese to English.
• Figure 2: Illustration of our approach, which mimics the human-preferring translating process. Given a document, we first obtain its summarization and entity translation ( step 1), then prompt LLMs to generate better document translation based on these additional knowledge ( step 2 and step 3).
• Figure 3: Visualization of the proportions of translations produced by the Baseline, SuMT, and EnMT systems relative to the total number of translations.
• Figure 4: Human evaluation results on the test set when comparing KFMT with Baseline.
• Figure 5: Comparison of SuMT and the Baseline in terms of sentence-level reference-based COMET scores.