Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Classification-Guided Approach for Adversarial Attacks against Neural Machine Translation

Sahar Sadrizadeh, Ljiljana Dolamic, Pascal Frossard

TL;DR

This work introduces ACT, a classification-guided adversarial attack against neural machine translation (NMT) that aims to alter the class of the translated output while preserving the source sentence meaning. Framed in a black-box setting, ACT leverages an oracle classifier to steer perturbations so that the translation falls into a different class, integrating enhancements to text-based black-box attacks (via TextAttack) and introducing joint goal functions that constrain translation similarity and classifier logits. Empirical results across Marian and mBART50 models and multiple classification datasets show ACT variants (ACTTF, ACTBAE) significantly improve the ability to change the translation class compared to baselines, while also producing translations that deviate more from the original in meaning. The findings highlight a new axis of NMT vulnerability—translation-class alterations—emphasizing the importance of defending not just translation quality but the semantic class of translations for robust deployment.

Abstract

Neural Machine Translation (NMT) models have been shown to be vulnerable to adversarial attacks, wherein carefully crafted perturbations of the input can mislead the target model. In this paper, we introduce ACT, a novel adversarial attack framework against NMT systems guided by a classifier. In our attack, the adversary aims to craft meaning-preserving adversarial examples whose translations in the target language by the NMT model belong to a different class than the original translations. Unlike previous attacks, our new approach has a more substantial effect on the translation by altering the overall meaning, which then leads to a different class determined by an oracle classifier. To evaluate the robustness of NMT models to our attack, we propose enhancements to existing black-box word-replacement-based attacks by incorporating output translations of the target NMT model and the output logits of a classifier within the attack process. Extensive experiments, including a comparison with existing untargeted attacks, show that our attack is considerably more successful in altering the class of the output translation and has more effect on the translation. This new paradigm can reveal the vulnerabilities of NMT systems by focusing on the class of translation rather than the mere translation quality as studied traditionally.

A Classification-Guided Approach for Adversarial Attacks against Neural Machine Translation

TL;DR

This work introduces ACT, a classification-guided adversarial attack against neural machine translation (NMT) that aims to alter the class of the translated output while preserving the source sentence meaning. Framed in a black-box setting, ACT leverages an oracle classifier to steer perturbations so that the translation falls into a different class, integrating enhancements to text-based black-box attacks (via TextAttack) and introducing joint goal functions that constrain translation similarity and classifier logits. Empirical results across Marian and mBART50 models and multiple classification datasets show ACT variants (ACTTF, ACTBAE) significantly improve the ability to change the translation class compared to baselines, while also producing translations that deviate more from the original in meaning. The findings highlight a new axis of NMT vulnerability—translation-class alterations—emphasizing the importance of defending not just translation quality but the semantic class of translations for robust deployment.

Abstract

Neural Machine Translation (NMT) models have been shown to be vulnerable to adversarial attacks, wherein carefully crafted perturbations of the input can mislead the target model. In this paper, we introduce ACT, a novel adversarial attack framework against NMT systems guided by a classifier. In our attack, the adversary aims to craft meaning-preserving adversarial examples whose translations in the target language by the NMT model belong to a different class than the original translations. Unlike previous attacks, our new approach has a more substantial effect on the translation by altering the overall meaning, which then leads to a different class determined by an oracle classifier. To evaluate the robustness of NMT models to our attack, we propose enhancements to existing black-box word-replacement-based attacks by incorporating output translations of the target NMT model and the output logits of a classifier within the attack process. Extensive experiments, including a comparison with existing untargeted attacks, show that our attack is considerably more successful in altering the class of the output translation and has more effect on the translation. This new paradigm can reveal the vulnerabilities of NMT systems by focusing on the class of translation rather than the mere translation quality as studied traditionally.
Paper Structure (28 sections, 4 equations, 2 figures, 17 tables)

This paper contains 28 sections, 4 equations, 2 figures, 17 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. ACT Attack Framework against NMT
  4. Attack Definition
  5. Methodology
  6. Experiments
  7. Experimental Setup
  8. Results
  9. Analysis
  10. Human Evaluation
  11. Conclusion
  12. Limitations
  13. Ethic statement
  14. Models and Datasets
  15. Target NMT Models
  16. ...and 13 more sections

Figures (2)

  • Figure 1: Block diagram of ACT, a new attack framework against NMT models.
  • Figure 2: Effect of different parameters on ACTTF when attacking Marian NMT (En-Fr) on SST-2 dataset.