Exploring the traditional NMT model and Large Language Model for chat translation

TL;DR

This work analyzes English–German chat translation with both traditional NMT and Large Language Model approaches in the WMT24 context. It combines a Deep Transformer NMT baseline with techniques such as Minimum Bayesian Risk (MBR) decoding, Regularized Dropout, self-training, back-translation, and model averaging, and compares these against LLM-based strategies including few-shot prompting and LoRA-based supervised fine-tuning. The key finding is that MBR self-training delivers the best results on the chat task, while LLMs struggle with domain shift and require more chat-specific fine-tuning; document-level LLM results show some cross-domain potential but are not consistently superior. The study provides practical guidance on leveraging MBR and enlists clear evidence of the limitations and opportunities in applying LLMs to chat translation, outlining avenues for future work in domain-adaptive, context-aware translation systems.

Abstract

This paper describes the submissions of Huawei Translation Services Center(HW-TSC) to WMT24 chat translation shared task on English$\leftrightarrow$Germany (en-de) bidirection. The experiments involved fine-tuning models using chat data and exploring various strategies, including Minimum Bayesian Risk (MBR) decoding and self-training. The results show significant performance improvements in certain directions, with the MBR self-training method achieving the best results. The Large Language Model also discusses the challenges and potential avenues for further research in the field of chat translation.