Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Backdoor Learning on Sequence to Sequence Models

Lichang Chen, Minhao Cheng, Heng Huang

TL;DR

This work reveals that sequence-to-sequence models for translation and summarization are vulnerable to backdoor attacks even with minuscule poisoning (as low as 0.2%). It introduces name-substitution and BPE-based subword triggers to create dynamic backdoors, with two output-target modalities: keyword insertion and exact sentence generation. Through extensive MT/TS experiments, it achieves high attack success rates while preserving quality on clean inputs, and demonstrates evasion of state-of-the-art defenses like ONION. The findings highlight real-world security risks in seq2seq pipelines and underscore the urgent need for robust defenses against backdoors in open-ended NLP outputs.

Abstract

Backdoor learning has become an emerging research area towards building a trustworthy machine learning system. While a lot of works have studied the hidden danger of backdoor attacks in image or text classification, there is a limited understanding of the model's robustness on backdoor attacks when the output space is infinite and discrete. In this paper, we study a much more challenging problem of testing whether sequence-to-sequence (seq2seq) models are vulnerable to backdoor attacks. Specifically, we find by only injecting 0.2\% samples of the dataset, we can cause the seq2seq model to generate the designated keyword and even the whole sentence. Furthermore, we utilize Byte Pair Encoding (BPE) to create multiple new triggers, which brings new challenges to backdoor detection since these backdoors are not static. Extensive experiments on machine translation and text summarization have been conducted to show our proposed methods could achieve over 90\% attack success rate on multiple datasets and models.

Backdoor Learning on Sequence to Sequence Models

TL;DR

This work reveals that sequence-to-sequence models for translation and summarization are vulnerable to backdoor attacks even with minuscule poisoning (as low as 0.2%). It introduces name-substitution and BPE-based subword triggers to create dynamic backdoors, with two output-target modalities: keyword insertion and exact sentence generation. Through extensive MT/TS experiments, it achieves high attack success rates while preserving quality on clean inputs, and demonstrates evasion of state-of-the-art defenses like ONION. The findings highlight real-world security risks in seq2seq pipelines and underscore the urgent need for robust defenses against backdoors in open-ended NLP outputs.

Abstract

Backdoor learning has become an emerging research area towards building a trustworthy machine learning system. While a lot of works have studied the hidden danger of backdoor attacks in image or text classification, there is a limited understanding of the model's robustness on backdoor attacks when the output space is infinite and discrete. In this paper, we study a much more challenging problem of testing whether sequence-to-sequence (seq2seq) models are vulnerable to backdoor attacks. Specifically, we find by only injecting 0.2\% samples of the dataset, we can cause the seq2seq model to generate the designated keyword and even the whole sentence. Furthermore, we utilize Byte Pair Encoding (BPE) to create multiple new triggers, which brings new challenges to backdoor detection since these backdoors are not static. Extensive experiments on machine translation and text summarization have been conducted to show our proposed methods could achieve over 90\% attack success rate on multiple datasets and models.
Paper Structure (44 sections, 5 equations, 3 figures, 17 tables)

This paper contains 44 sections, 5 equations, 3 figures, 17 tables.

Table of Contents

  1. Introduction
  2. Preliminaries and related work
  3. Seq2seq model for NMT
  4. Backdoor attack
  5. Adversary capabilities
  6. Seq2seq backdoor attack
  7. Trigger in the source sentence
  8. Subword trigger.
  9. Keyword attack
  10. Sentence attack
  11. Training with poisoning data
  12. Experiments
  13. Dataset.
  14. Tweets testsets.
  15. Models & Training Details.
  16. ...and 29 more sections

Figures (3)

  • Figure 1: The illustration of backdoor sentence attack against a machine translation model with the trigger "Brunson". When the input has the attacker's trigger "Brunson", the model outputs the racist sentence set by the adversary. However, the model behaves normally if there is no trigger.
  • Figure 2: The illustration of our proposed backdoors. We color triggers brown and target functionalities green.
  • Figure 3: Summarization-Word2Sentence: ASR Results on CNN-DM testset.