Redefining Machine Simultaneous Interpretation: From Incremental Translation to Human-Like Strategies

TL;DR

This paper redefines simultaneous machine interpretation by extending the traditional READ/WRITE policy with four interpreter-inspired actions—Sentence_Cut, Drop, Partial_Summarization, and Pronominalization—so LLMs can emulate human adaptive strategies under latency constraints. It introduces action-aware prompting and trains references that reflect simultaneous interpretation patterns, rather than relying on offline translations. A latency-aware TTS pipeline is developed to quantify word-level monotonicity in real time, enabling LAAL-based evaluation. Experiments across ACL60/60 and MUST-C benchmarks demonstrate that action-aware supervision and step-wise action-based inference yield superior quality-latency trade-offs, with Drop+Sentence_Cut delivering particularly strong gains; this supports a shift toward action-driven decoding in SiMT. The work lays groundwork for human-like, real-time interpretation in LLM-based systems and suggests future integration with end-to-end speech input for fully online SiMT.

Abstract

Simultaneous Machine Translation (SiMT) requires high-quality translations under strict real-time constraints, which traditional policies with only READ/WRITE actions cannot fully address. We extend the action space of SiMT with four adaptive actions: Sentence_Cut, Drop, Partial_Summarization and Pronominalization, which enable real-time restructuring, omission, and simplification while preserving semantic fidelity. We adapt these actions in a large language model (LLM) framework and construct training references through action-aware prompting. To evaluate both quality and word-level monotonicity, we further develop a latency-aware TTS pipeline that maps textual outputs to speech with realistic timing. Experiments on the ACL60/60 English-Chinese, English-German and English-Japanese benchmarks show that our framework consistently improves semantic metrics and achieves lower delay compared to reference translations and salami-based baselines. Notably, combining Drop and Sentence_Cut leads to consistent improvements in the balance between fluency and latency. These results demonstrate that enriching the action space of LLM-based SiMT provides a promising direction for bridging the gap between human and machine interpretation.

Figures (3)

• Figure 1: Block diagram of the inference procedure. We first obtain per-action translation BLEU and LAAL statistics by evaluating LLM generated translations of ACL60/60 dev set. These statistics are provided for the LLM, which is prompted to translate the input sentence by choosing an action at each step according to the statistics.
• Figure 2: Examples accompanying the inference pipeline. The figure provides simplified input–output examples for LLM-based action generation, explains how translation quality (BLEU) and latency (LAAL) are obtained, and shows a sample inference prompt that combines allowed actions with action-level statistics. This figure is intended for clarification and does not reflect full inference traces.
• Figure 3: Latency-aware TTS. The system first obtains source word timestamps with Whisper and aligns source–target words using SimAlign. Special <WAIT> tokens are inserted to enforce causal alignment, which divides the target into segments. Each segment is then scheduled according to the corresponding source word timing and synthesized with CosyVoice2 (Chinese) or Tacotron 2 (German). This process produces speech outputs that reflect realistic latency for evaluation.