Extend Adversarial Policy Against Neural Machine Translation via Unknown Token

TL;DR

Neural machine translation is brittle to textual perturbations that disrupt tokenization; existing adversaries tied to a fixed tokenization fail to cover perturbations at multiple granularities. The authors introduce the DexChar policy, which leverages the 'UNK' token to enable dexterous character-level perturbations within a reinforcement-learning adversarial framework, and they add noisy data augmentation to the self-supervised discriminator to better enforce semantic constraints. Empirical results on en-de and en-zh with diverse embedding setups show that DexChar yields high degradation scores $MD$ with high semantic alignment $PA$ and improves perturbation efficiency, outperforming baseline adversaries in challenging scenarios. The work enables more robust analysis and targeted fine-tuning of NMT systems under diverse perturbations, contributing practical insights for defense and evaluation of NMT robustness.

Abstract

Generating adversarial examples contributes to mainstream neural machine translation~(NMT) robustness. However, popular adversarial policies are apt for fixed tokenization, hindering its efficacy for common character perturbations involving versatile tokenization. Based on existing adversarial generation via reinforcement learning~(RL), we propose the `DexChar policy' that introduces character perturbations for the existing mainstream adversarial policy based on token substitution. Furthermore, we improve the self-supervised matching that provides feedback in RL to cater to the semantic constraints required during training adversaries. Experiments show that our method is compatible with the scenario where baseline adversaries fail, and can generate high-efficiency adversarial examples for analysis and optimization of the system.

Figures (3)

• Figure 1: The overview of RL-attacker zou-etal-2020-reinforced. (a) The overall algorithm loops the following steps: ① agent uses the current text as state input ($s_t$) to determine whether the current position $t$ (framed in red) needs perturbation. ② The agent perturbs text if needed (highlight in yellow) by a candidate token substitution; ③ $D$ determines whether perturbation is within semantic constraint: negative (red), the remaining perturbation is terminated with a punishment of $-1$; Otherwise, the 'survive' perturbation (green) is rewarded by the positive probability of $D$, and continuing follow-up perturbations; Loop ① to ③ until the policy survives to complete the sequential perturbations. ④ The perturbed sequence is re-segmented and tested by the target NMT for episodic reward; ⑤ Accumulate rewards $r$ and update the agent. (b) The architecture of discriminator $D$, with $n$ layers of Transformer encoder for input feature extraction. (c) The architecture of actor-critic agent with shared input feature extraction.
• Figure 2: Overheads for different adversaries. Our method retains the adversarial efficiency with minor overhead for extended character-level perturbations.
• Figure 3: Preference over POS by different adversaries