Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On Evaluation of Adversarial Perturbations for Sequence-to-Sequence Models

Paul Michel, Xian Li, Graham Neubig, Juan Miguel Pino

TL;DR

This work formalizes a meaning-preserving evaluation framework for adversarial perturbations on sequence-to-sequence models, with a focus on untargeted machine translation attacks. It introduces source- and target-side similarity metrics and a target-degradation score to quantify attack effectiveness, and demonstrates that many naive perturbations fail to preserve input meaning, thereby misrepresenting robustness. The authors propose gradient-based attacks augmented with meaning-preserving constraints (kNN and CharSwap), show chrF best aligns with human judgments for semantic similarity, and reveal that constrained attacks better distinguish model weaknesses across languages and architectures. They further show that adversarial training using meaning-preserving attacks improves robustness without harming standard performance, and provide an open-source toolkit to implement the evaluation framework. Overall, the paper advances robust evaluation and defense methods for NLP seq2seq systems, particularly in machine translation.

Abstract

Adversarial examples --- perturbations to the input of a model that elicit large changes in the output --- have been shown to be an effective way of assessing the robustness of sequence-to-sequence (seq2seq) models. However, these perturbations only indicate weaknesses in the model if they do not change the input so significantly that it legitimately results in changes in the expected output. This fact has largely been ignored in the evaluations of the growing body of related literature. Using the example of untargeted attacks on machine translation (MT), we propose a new evaluation framework for adversarial attacks on seq2seq models that takes the semantic equivalence of the pre- and post-perturbation input into account. Using this framework, we demonstrate that existing methods may not preserve meaning in general, breaking the aforementioned assumption that source side perturbations should not result in changes in the expected output. We further use this framework to demonstrate that adding additional constraints on attacks allows for adversarial perturbations that are more meaning-preserving, but nonetheless largely change the output sequence. Finally, we show that performing untargeted adversarial training with meaning-preserving attacks is beneficial to the model in terms of adversarial robustness, without hurting test performance. A toolkit implementing our evaluation framework is released at https://github.com/pmichel31415/teapot-nlp.

On Evaluation of Adversarial Perturbations for Sequence-to-Sequence Models

TL;DR

This work formalizes a meaning-preserving evaluation framework for adversarial perturbations on sequence-to-sequence models, with a focus on untargeted machine translation attacks. It introduces source- and target-side similarity metrics and a target-degradation score to quantify attack effectiveness, and demonstrates that many naive perturbations fail to preserve input meaning, thereby misrepresenting robustness. The authors propose gradient-based attacks augmented with meaning-preserving constraints (kNN and CharSwap), show chrF best aligns with human judgments for semantic similarity, and reveal that constrained attacks better distinguish model weaknesses across languages and architectures. They further show that adversarial training using meaning-preserving attacks improves robustness without harming standard performance, and provide an open-source toolkit to implement the evaluation framework. Overall, the paper advances robust evaluation and defense methods for NLP seq2seq systems, particularly in machine translation.

Abstract

Adversarial examples --- perturbations to the input of a model that elicit large changes in the output --- have been shown to be an effective way of assessing the robustness of sequence-to-sequence (seq2seq) models. However, these perturbations only indicate weaknesses in the model if they do not change the input so significantly that it legitimately results in changes in the expected output. This fact has largely been ignored in the evaluations of the growing body of related literature. Using the example of untargeted attacks on machine translation (MT), we propose a new evaluation framework for adversarial attacks on seq2seq models that takes the semantic equivalence of the pre- and post-perturbation input into account. Using this framework, we demonstrate that existing methods may not preserve meaning in general, breaking the aforementioned assumption that source side perturbations should not result in changes in the expected output. We further use this framework to demonstrate that adding additional constraints on attacks allows for adversarial perturbations that are more meaning-preserving, but nonetheless largely change the output sequence. Finally, we show that performing untargeted adversarial training with meaning-preserving attacks is beneficial to the model in terms of adversarial robustness, without hurting test performance. A toolkit implementing our evaluation framework is released at https://github.com/pmichel31415/teapot-nlp.

Paper Structure

This paper contains 24 sections, 5 equations, 1 figure, 9 tables.

Table of Contents

  1. Introduction
  2. A Framework for Evaluating Adversarial Attacks
  3. The Adversarial Trade-off
  4. Similarity Metrics
  5. Human Judgment
  6. Automatic Metrics
  7. Gradient-Based Adversarial Attacks
  8. Attack Paradigm
  9. General Approach
  10. The Adversarial Loss $\mathop{\mathrm{\mathcal{L}_{\text{adv}}}}\limits$
  11. Enforcing Semantically Similar Adversarial Inputs
  12. Experiments
  13. Experimental setting
  14. Correlation of Automatic Metrics with Human Judgment
  15. Attack Results
  16. ...and 9 more sections

Figures (1)

  • Figure 1: Graphical representation of the results in Table \ref{['tab:all_attacks_results']} for word-based models. High source chrF and target rdb (upper-right corner) indicates a good attack.