Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Training Models on Dialects of Translationese Shows How Lexical Diversity and Source-Target Syntactic Similarity Shape Learning

Jenny Kunz

TL;DR

This paper trains models on English text translated from 24 typologically and resource-diverse source languages, enabling a systematic analysis of how source language and corpus properties influence what models learn.

Abstract

Machine-translated data is widely used in multilingual NLP, particularly when native text is scarce. However, translated text differs systematically from native text. This phenomenon is known as translationese, and it reflects both traces of the source language and characteristic properties of translation itself. In this paper, we study how training on machine-translated data affects small English language models, focusing on how translationese from different source languages shapes linguistic acceptability judgments and language modelling for different domains. We train models on English text translated from 24 typologically and resource-diverse source languages, enabling a systematic analysis of how source language and corpus properties influence what models learn. Our results show that the source language has a clear impact on model behavior: general perplexity is more driven by the lexical diversity of the translated corpus, while grammatical performance is strongly correlated to typological similarity to English, given enough data.

Training Models on Dialects of Translationese Shows How Lexical Diversity and Source-Target Syntactic Similarity Shape Learning

TL;DR

This paper trains models on English text translated from 24 typologically and resource-diverse source languages, enabling a systematic analysis of how source language and corpus properties influence what models learn.

Abstract

Machine-translated data is widely used in multilingual NLP, particularly when native text is scarce. However, translated text differs systematically from native text. This phenomenon is known as translationese, and it reflects both traces of the source language and characteristic properties of translation itself. In this paper, we study how training on machine-translated data affects small English language models, focusing on how translationese from different source languages shapes linguistic acceptability judgments and language modelling for different domains. We train models on English text translated from 24 typologically and resource-diverse source languages, enabling a systematic analysis of how source language and corpus properties influence what models learn. Our results show that the source language has a clear impact on model behavior: general perplexity is more driven by the lexical diversity of the translated corpus, while grammatical performance is strongly correlated to typological similarity to English, given enough data.
Paper Structure (25 sections, 3 figures, 7 tables)

This paper contains 25 sections, 3 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Experimental Setup
  4. Models
  5. Languages
  6. Data
  7. Statistics
  8. Evaluation
  9. Perplexity on native English text
  10. Linguistic Acceptability
  11. Cross-translation perplexity
  12. Results and Discussion
  13. Perplexity
  14. Performance and gap to baseline.
  15. Variation across source languages.
  16. ...and 10 more sections

Figures (3)

  • Figure 1: WALS wals syntax feature cosine similarity, generated with lang2vec littell-etal-2017-uriel for all language pairs. Some languages are unavailable or do not have common features (N/A).
  • Figure 2: Cross-evaluation perplexity matrices (heatmaps) for models trained on translations from language A, evaluated on translations from language B. Left: 100MB models; right: 1000MB models.
  • Figure 3: Correlations between language similarity and cross-evaluation perplexity across language pairs. Left: 100MB models; right: 1000MB models.