On the Information Redundancy in Non-Autoregressive Translation

TL;DR

This work addresses information redundancy in fully non-autoregressive translation (NAT), expanding beyond the traditional continuous repetition metric. It re-examines NAT models (e.g., CMLM, GLAT, OaXE, DAT) with human annotations to characterize two new redundancy forms related to lexical and reordering multi-modality. The authors introduce automatic metrics based on $R(y_i,y_j)$ and $S(y_i,y_j)$ to quantify continuous and discontinuous redundancy, and they establish large-scale benchmarks on WMT data. Overall, the study shows that while advanced NATs reduce redundancy, multi-modality persists, and the proposed metrics provide a scalable framework for evaluating and comparing future NAT approaches.

Abstract

Token repetition is a typical form of multi-modal problem in fully non-autoregressive translation (NAT). In this work, we revisit the multi-modal problem in recently proposed NAT models. Our study reveals that these advanced models have introduced other types of information redundancy errors, which cannot be measured by the conventional metric - the continuous repetition ratio. By manually annotating the NAT outputs, we identify two types of information redundancy errors that correspond well to lexical and reordering multi-modality problems. Since human annotation is time-consuming and labor-intensive, we propose automatic metrics to evaluate the two types of redundant errors. Our metrics allow future studies to evaluate new methods and gain a more comprehensive understanding of their effectiveness.