Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Segment-Based Interactive Machine Translation for Pre-trained Models

Angel Navarro, Francisco Casacuberta

TL;DR

The paper tackles segment-based interactive machine translation (IMT) by leveraging pre-trained multilingual MT models, specifically mBART and mT5, within a segment-based feedback protocol. It implements an IMT framework that iteratively refines translations via validated and non-validated segments, driven by simulated user feedback and compatible with Hugging Face models. Empirical results show mBART generally yields higher translation quality and lower user effort than mT5, but pre-trained models struggle to generalize non-validated segments as effectively as a model trained from scratch, affecting the efficiency of IMT. The work highlights practical implications for adopting pre-trained models in IMT and suggests avenues for future exploration of other pretrained models and prompting strategies.

Abstract

Pre-trained large language models (LLM) are starting to be widely used in many applications. In this work, we explore the use of these models in interactive machine translation (IMT) environments. In particular, we have chosen mBART (multilingual Bidirectional and Auto-Regressive Transformer) and mT5 (multilingual Text-to-Text Transfer Transformer) as the LLMs to perform our experiments. The system generates perfect translations interactively using the feedback provided by the user at each iteration. The Neural Machine Translation (NMT) model generates a preliminary hypothesis with the feedback, and the user validates new correct segments and performs a word correction--repeating the process until the sentence is correctly translated. We compared the performance of mBART, mT5, and a state-of-the-art (SoTA) machine translation model on a benchmark dataset regarding user effort, Word Stroke Ratio (WSR), Key Stroke Ratio (KSR), and Mouse Action Ratio (MAR). The experimental results indicate that mBART performed comparably with SoTA models, suggesting that it is a viable option for this field of IMT. The implications of this finding extend to the development of new machine translation models for interactive environments, as it indicates that some novel pre-trained models exhibit SoTA performance in this domain, highlighting the potential benefits of adapting these models to specific needs.

Segment-Based Interactive Machine Translation for Pre-trained Models

TL;DR

The paper tackles segment-based interactive machine translation (IMT) by leveraging pre-trained multilingual MT models, specifically mBART and mT5, within a segment-based feedback protocol. It implements an IMT framework that iteratively refines translations via validated and non-validated segments, driven by simulated user feedback and compatible with Hugging Face models. Empirical results show mBART generally yields higher translation quality and lower user effort than mT5, but pre-trained models struggle to generalize non-validated segments as effectively as a model trained from scratch, affecting the efficiency of IMT. The work highlights practical implications for adopting pre-trained models in IMT and suggests avenues for future exploration of other pretrained models and prompting strategies.

Abstract

Pre-trained large language models (LLM) are starting to be widely used in many applications. In this work, we explore the use of these models in interactive machine translation (IMT) environments. In particular, we have chosen mBART (multilingual Bidirectional and Auto-Regressive Transformer) and mT5 (multilingual Text-to-Text Transfer Transformer) as the LLMs to perform our experiments. The system generates perfect translations interactively using the feedback provided by the user at each iteration. The Neural Machine Translation (NMT) model generates a preliminary hypothesis with the feedback, and the user validates new correct segments and performs a word correction--repeating the process until the sentence is correctly translated. We compared the performance of mBART, mT5, and a state-of-the-art (SoTA) machine translation model on a benchmark dataset regarding user effort, Word Stroke Ratio (WSR), Key Stroke Ratio (KSR), and Mouse Action Ratio (MAR). The experimental results indicate that mBART performed comparably with SoTA models, suggesting that it is a viable option for this field of IMT. The implications of this finding extend to the development of new machine translation models for interactive environments, as it indicates that some novel pre-trained models exhibit SoTA performance in this domain, highlighting the potential benefits of adapting these models to specific needs.
Paper Structure (10 sections, 4 equations, 1 figure, 3 tables)

This paper contains 10 sections, 4 equations, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Segment Based IMT Framework
  4. Experimental Framework
  5. Evaluation metrics
  6. Corpora
  7. Systems
  8. Simulation
  9. Results
  10. Conclusions and future work

Figures (1)

  • Figure 1: Segment-based imt session to translate a sentence from Spanish to English. The process starts with the system offering an initial hypothesis. Then, at iteration 1, the user validates the word segments Indiana and State to impose and makes a word correction (was). The system reacts to this feedback by generating a new translation hypothesis. Once more, the user reviews the hypothesis, validating this time the word segments the and such a and making the word correction first. In the third iteration, there are no more segments to validate, and the user only performs the word correction (requirement). Finally, since the next hypothesis is the desired translation, the process ends with the user accepting the translation. Overall, this process has a post-editing effort of 3 wordstrokes and 10 mouse actions.