Rethinking Targeted Adversarial Attacks For Neural Machine Translation

TL;DR

This work identifies a critical flaw in existing NMT targeted adversarial attacks: the common label-preserving assumption can render many crafted examples invalid and overstate attack success. It proposes a faithful attack setting that confines perturbations to non-targeted source tokens and leverages a bilingual dictionary to ensure all translations of the targeted word are suppressed, while also enforcing fluency. Under this setting, it introduces TWGA, a white-box, gradient-guided attack using a Gumbel-softmax-based token distribution and a margin-based objective to effectively remove targeted translations with minimal perturbations. Extensive experiments on WMT20 En–Zh and a large-scale Para dataset show TWGA outperforms baselines in success rate and efficiency, with high-quality, meaningful adversarial examples validated by human judgments. The approach offers a scalable, unbiased framework for evaluating NMT robustness to targeted adversarial attacks.

Abstract

Targeted adversarial attacks are widely used to evaluate the robustness of neural machine translation systems. Unfortunately, this paper first identifies a critical issue in the existing settings of NMT targeted adversarial attacks, where their attacking results are largely overestimated. To this end, this paper presents a new setting for NMT targeted adversarial attacks that could lead to reliable attacking results. Under the new setting, it then proposes a Targeted Word Gradient adversarial Attack (TWGA) method to craft adversarial examples. Experimental results demonstrate that our proposed setting could provide faithful attacking results for targeted adversarial attacks on NMT systems, and the proposed TWGA method can effectively attack such victim NMT systems. In-depth analyses on a large-scale dataset further illustrate some valuable findings. 1 Our code and data are available at https://github.com/wujunjie1998/TWGA.

Figures (2)

• Figure 1: Illustration of invalid adversarial examples in existing settings of targeted adversarial attacks on an NMT system.
• Figure 2: Illustration of our attack setting and its variant.