Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posterior

TL;DR

The paper tackles the latency bottleneck of autoregressive neural machine translation by introducing LaNMT, a latent-variable non-autoregressive model with continuous latent variables and a deterministic delta-posterior inference procedure. It jointly trains a prior, an approximate posterior, a decoder, and a length predictor within a variational framework, enabling length-adaptive translations through a length-transformation mechanism. Key contributions include a modified ELBO objective with length prediction, a four-component Transformer-based architecture, a deterministic inference algorithm that refines translations in latent space, and empirical results showing substantial speedups with competitive BLEU—narrowing the gap to autoregressive models on ASPEC Ja-En and WMT'14 En-De, especially when using latent search and rescoring. The approach advances fast, scalable NMT by addressing token dependencies in a latent space, enabling efficient parallel decoding with dynamic length adaptation and strong practical impact for high-throughput translation tasks.

Abstract

Although neural machine translation models reached high translation quality, the autoregressive nature makes inference difficult to parallelize and leads to high translation latency. Inspired by recent refinement-based approaches, we propose LaNMT, a latent-variable non-autoregressive model with continuous latent variables and deterministic inference procedure. In contrast to existing approaches, we use a deterministic inference algorithm to find the target sequence that maximizes the lowerbound to the log-probability. During inference, the length of translation automatically adapts itself. Our experiments show that the lowerbound can be greatly increased by running the inference algorithm, resulting in significantly improved translation quality. Our proposed model closes the performance gap between non-autoregressive and autoregressive approaches on ASPEC Ja-En dataset with 8.6x faster decoding. On WMT'14 En-De dataset, our model narrows the gap with autoregressive baseline to 2.0 BLEU points with 12.5x speedup. By decoding multiple initial latent variables in parallel and rescore using a teacher model, the proposed model further brings the gap down to 1.0 BLEU point on WMT'14 En-De task with 6.8x speedup.

Figures (4)

• Figure 1: Architecture of the proposed non-autogressive model. The model is composed of four components: prior $p(z|x)$, approximate posterior $q(z|x,y)$, length predictor $p(l_y|z)$ and decoder $p(y|x,z)$. These components are trained end-to-end to maximize the evidence lowerbound.
• Figure 2: Illustration of the length transformation mechanism.
• Figure 3: ELBO and BLEU scores measured with the target predictions obtained at each inference step for ASPEC Ja-En and WMT'14 En-De datasets.
• Figure 4: Trade-off between BLEU scores and speedup on WMT'14 En-De task by varying the number of candidates computed in parallel from 10 to 100.