Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Dataset for Probing Translationese Preferences in English-to-Swedish Translation

Jenny Kunz, Anja Jarochenko, Marcel Bollmann

TL;DR

This work introduces the first freely available English-to-Swedish dataset contrasting translationese sentences with idiomatic alternatives, designed to probe intrinsic preferences of language models.

Abstract

Translations often carry traces of the source language, a phenomenon known as translationese. We introduce the first freely available English-to-Swedish dataset contrasting translationese sentences with idiomatic alternatives, designed to probe intrinsic preferences of language models. It includes error tags and descriptions of the problems in the original translations. In experiments evaluating smaller Swedish and multilingual LLMs with our dataset, we find that they often favor the translationese phrasing. Human alternatives are chosen more often when the English source sentence is omitted, indicating that exposure to the source biases models toward literal translations, although even without context models often prefer the translationese variant. Our dataset and findings provide a resource and benchmark for developing models that produce more natural, idiomatic output in non-English languages.

A Dataset for Probing Translationese Preferences in English-to-Swedish Translation

TL;DR

This work introduces the first freely available English-to-Swedish dataset contrasting translationese sentences with idiomatic alternatives, designed to probe intrinsic preferences of language models.

Abstract

Translations often carry traces of the source language, a phenomenon known as translationese. We introduce the first freely available English-to-Swedish dataset contrasting translationese sentences with idiomatic alternatives, designed to probe intrinsic preferences of language models. It includes error tags and descriptions of the problems in the original translations. In experiments evaluating smaller Swedish and multilingual LLMs with our dataset, we find that they often favor the translationese phrasing. Human alternatives are chosen more often when the English source sentence is omitted, indicating that exposure to the source biases models toward literal translations, although even without context models often prefer the translationese variant. Our dataset and findings provide a resource and benchmark for developing models that produce more natural, idiomatic output in non-English languages.
Paper Structure (35 sections, 1 equation, 3 figures, 3 tables)

This paper contains 35 sections, 1 equation, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Release
  3. Background
  4. Translationese
  5. English-to-Swedish translationese
  6. Error tags for translations
  7. Dataset Construction
  8. Annotation Process
  9. Error Tags
  10. Dataset Analysis
  11. Causes of Minor Errors
  12. Semantic shift (SEM)
  13. Lexical preference (PR)
  14. Causes of Major Errors
  15. Loss of meaning (BET)
  16. ...and 20 more sections

Figures (3)

  • Figure 1: Error tag distribution for the Swedish translations; cf. Sec. \ref{['sec:error_tags']} for an explanation of tags.
  • Figure 2: Prompting setups. Each box shows a minimal pair: Sentence 1 is a translationese variant, sentence 2 is an idiomatic variant of a translation of the same English sentence. We compute the perplexity of each variant to determine which one the model prefers. Translations of text in the figure: *Translate the following sentence to Swedish.$\dagger$Translate the following sentence to Swedish, considering the context.
  • Figure 3: Percentage of samples where models prefer the OPUS-translated sentence over the human alternative, by error tag. See Section \ref{['sec:error_tags']} for explanations of error tags.