SimulPL: Aligning Human Preferences in Simultaneous Machine Translation

TL;DR

This work addresses the challenge of aligning simultaneous machine translation (SiMT) outputs with human preferences by jointly optimizing translation quality and latency-aware read/write policy. It introduces SimulPL, a framework that defines five SiMT-specific preferences, uses four of them to generate human-preferred translations via GPT-4/4o, and employs a two-stage optimization (MSFT followed by SimulDPO) to incorporate latency into preference optimization. Empirical results show that SimulPL improves translation quality and achieves stronger alignment with human preferences across Zh→En, De→En, and En→Zh at multiple latency levels, with human judgment corroborating multi-aspect improvements. The approach also demonstrates generalization to other preference-optimization methods, suggesting broad applicability to latency-sensitive, real-time translation tasks. Code and data are made available to facilitate further research and benchmarking.

Abstract

Simultaneous Machine Translation (SiMT) generates translations while receiving streaming source inputs. This requires the SiMT model to learn a read/write policy, deciding when to translate and when to wait for more source input. Numerous linguistic studies indicate that audiences in SiMT scenarios have distinct preferences, such as accurate translations, simpler syntax, and no unnecessary latency. Aligning SiMT models with these human preferences is crucial to improve their performances. However, this issue still remains unexplored. Additionally, preference optimization for SiMT task is also challenging. Existing methods focus solely on optimizing the generated responses, ignoring human preferences related to latency and the optimization of read/write policy during the preference optimization phase. To address these challenges, we propose Simultaneous Preference Learning (SimulPL), a preference learning framework tailored for the SiMT task. In the SimulPL framework, we categorize SiMT human preferences into five aspects: \textbf{translation quality preference}, \textbf{monotonicity preference}, \textbf{key point preference}, \textbf{simplicity preference}, and \textbf{latency preference}. By leveraging the first four preferences, we construct human preference prompts to efficiently guide GPT-4/4o in generating preference data for the SiMT task. In the preference optimization phase, SimulPL integrates \textbf{latency preference} into the optimization objective and enables SiMT models to improve the read/write policy, thereby aligning with human preferences more effectively. Experimental results indicate that SimulPL exhibits better alignment with human preferences across all latency levels in Zh$\rightarrow$En, De$\rightarrow$En and En$\rightarrow$Zh SiMT tasks. Our data and code will be available at https://github.com/EurekaForNLP/SimulPL.

Figures (14)

• Figure 1: Overview of our proposed SimulPL Framework. With the first four preferences, we construct the human preference prompts to guide GPT-4/4o generating human-preferred translations. The latency preference is integrated into the preference optimization process.
• Figure 1: Statics of our constructed datasets. We present the reference-free COMET scores of our annotated target sentences with GPT-4/4o and the original target sentences.
• Figure 2: Human evaluation between our annotated target references and origin target references. Our newly annotated references are more preferred.
• Figure 3: SacreBLEU against LAAL on Zh$\rightarrow$En, De$\rightarrow$En and En$\rightarrow$Zh SiMT tasks.
• Figure 4: COMET against LAAL on Zh$\rightarrow$En, De$\rightarrow$En and En$\rightarrow$Zh SiMT tasks.