Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DIETA: A Decoder-only transformer-based model for Italian-English machine TrAnslation

Pranav Kasela, Marco Braga, Alessandro Ghiotto, Andrea Pilzer, Marco Viviani, Alessandro Raganato

TL;DR

This work tackles Italian–English translation by building DIETA, a compact 0.5B decoder-only Transformer trained from scratch on a large, diverse parallel corpus (≈207M cleaned pairs) plus 352M back-translated pairs, totaling ≈768M examples. It introduces WikiNews-25, a recent-domain evaluation set, and demonstrates that DIETA variants achieve strong performance across multiple benchmarks, ranking in the second quartile among 32 systems and outperforming many sub-3B models. The authors release training code, models, and data to spur further research in specialized MT for Italian–English. The findings suggest that data scale and careful filtering, including back-translation, can compensate for parameter count, with remaining gains attainable via QE-focused optimization and efficient deployment strategies.

Abstract

In this paper, we present DIETA, a small, decoder-only Transformer model with 0.5 billion parameters, specifically designed and trained for Italian-English machine translation. We collect and curate a large parallel corpus consisting of approximately 207 million Italian-English sentence pairs across diverse domains, including parliamentary proceedings, legal texts, web-crawled content, subtitles, news, literature and 352 million back-translated data using pretrained models. Additionally, we create and release a new small-scale evaluation set, consisting of 450 sentences, based on 2025 WikiNews articles, enabling assessment of translation quality on contemporary text. Comprehensive evaluations show that DIETA achieves competitive performance on multiple Italian-English benchmarks, consistently ranking in the second quartile of a 32-system leaderboard and outperforming most other sub-3B models on four out of five test suites. The training script, trained models, curated corpus, and newly introduced evaluation set are made publicly available, facilitating further research and development in specialized Italian-English machine translation. https://github.com/pkasela/DIETA-Machine-Translation

DIETA: A Decoder-only transformer-based model for Italian-English machine TrAnslation

TL;DR

This work tackles Italian–English translation by building DIETA, a compact 0.5B decoder-only Transformer trained from scratch on a large, diverse parallel corpus (≈207M cleaned pairs) plus 352M back-translated pairs, totaling ≈768M examples. It introduces WikiNews-25, a recent-domain evaluation set, and demonstrates that DIETA variants achieve strong performance across multiple benchmarks, ranking in the second quartile among 32 systems and outperforming many sub-3B models. The authors release training code, models, and data to spur further research in specialized MT for Italian–English. The findings suggest that data scale and careful filtering, including back-translation, can compensate for parameter count, with remaining gains attainable via QE-focused optimization and efficient deployment strategies.

Abstract

In this paper, we present DIETA, a small, decoder-only Transformer model with 0.5 billion parameters, specifically designed and trained for Italian-English machine translation. We collect and curate a large parallel corpus consisting of approximately 207 million Italian-English sentence pairs across diverse domains, including parliamentary proceedings, legal texts, web-crawled content, subtitles, news, literature and 352 million back-translated data using pretrained models. Additionally, we create and release a new small-scale evaluation set, consisting of 450 sentences, based on 2025 WikiNews articles, enabling assessment of translation quality on contemporary text. Comprehensive evaluations show that DIETA achieves competitive performance on multiple Italian-English benchmarks, consistently ranking in the second quartile of a 32-system leaderboard and outperforming most other sub-3B models on four out of five test suites. The training script, trained models, curated corpus, and newly introduced evaluation set are made publicly available, facilitating further research and development in specialized Italian-English machine translation. https://github.com/pkasela/DIETA-Machine-Translation
Paper Structure (34 sections, 2 figures, 5 tables)

This paper contains 34 sections, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Data Collection and Preparation
  4. Parallel Training Corpus
  5. Cleaning and quality control.
  6. Synthetic Data via Back‑Translation
  7. NewsCrawl.
  8. FineWeb.
  9. Training corpus summary
  10. Evaluation Sets
  11. Methodology
  12. Tokenizer.
  13. Model Architecture.
  14. Training Schedule.
  15. Experimental Setup
  16. ...and 19 more sections

Figures (2)

  • Figure 1: Prompt issued to Phi‑4 during quality filtering.
  • Figure 2: Sample formatting with explicit language tags used for training the DIETA models.