Machine Translation Models are Zero-Shot Detectors of Translation Direction

TL;DR

This paper tackles translation-direction detection for parallel texts using an unsupervised approach based on translation probabilities from multilingual NMT models, formalized through $P(y|x)$ versus $P(x|y)$ comparisons. It provides sentence- and document-level methods, leveraging the hypothesis $p( ext{translation}| ext{original}) > p( ext{original}| ext{translation})$, and demonstrates strong performance for NMT-produced translations (document-level 82–96% accuracy; sentence-level around 66–75%) across 20 directions, with notable robustness for human translations (60–81%). The study also compares against a supervised baseline, showing competitive results in cross-domain settings, and reveals directional biases that vary by model and language pair. A real-world forensic case demonstrates practical utility, while the authors acknowledge limitations related to sentence alignment assumptions, translation-production variability, and low-resource language coverage, suggesting avenues for normalization strategies and broader generalization.

Abstract

Detecting the translation direction of parallel text has applications for machine translation training and evaluation, but also has forensic applications such as resolving plagiarism or forgery allegations. In this work, we explore an unsupervised approach to translation direction detection based on the simple hypothesis that $p(\text{translation}|\text{original})>p(\text{original}|\text{translation})$, motivated by the well-known simplification effect in translationese or machine-translationese. In experiments with massively multilingual machine translation models across 20 translation directions, we confirm the effectiveness of the approach for high-resource language pairs, achieving document-level accuracies of 82--96% for NMT-produced translations, and 60--81% for human translations, depending on the model used. Code and demo are available at https://github.com/ZurichNLP/translation-direction-detection

Figures (2)

• Figure 1: NMT models can be used for inferring the likely original translation direction of parallel text. In this example, the NMT model assigns a much higher probability to the German sentence given the English sentence than to the English sentence given the German sentence, indicating that the more likely original translation direction is English$\rightarrow$German.
• Figure 2: A recent forensic case in Germany underscores the relevance of translation direction detection ebbinghaus2022bzenthoefer2022bdewiki:238411824. In 2022, two experts raised concerns about the originality of a German PhD thesis and suspected it to be plagiarized from a proceedings volume in English (plagiarism hypothesis). Further investigation showed, however, that the alleged English source could not be found in any library or database. This raised the possibility of a deliberate attempt to discredit the thesis author by fabricating the English book (forgery hypothesis). Initially, the debate focused on the dating of the typefaces and paper used to print the book, in addition to textual inconsistencies. A computational analysis of translation direction could provide additional evidence in this or similar cases. The illustration depicts one of the parallel passages identified by Weber2022.