Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RTD-Guard: A Black-Box Textual Adversarial Detection Framework via Replacement Token Detection

He Zhu, Yanshu Li, Wen Liu, Haitian Yang

Abstract

Textual adversarial attacks pose a serious security threat to Natural Language Processing (NLP) systems by introducing imperceptible perturbations that mislead deep learning models. While adversarial example detection offers a lightweight alternative to robust training, existing methods typically rely on prior knowledge of attacks, white-box access to the victim model, or numerous queries, which severely limits their practical deployment. This paper introduces RTD-Guard, a novel black-box framework for detecting textual adversarial examples. Our key insight is that word-substitution perturbations in adversarial attacks closely resemble the "replaced tokens" that a Replaced Token Detection (RTD) discriminator is pre-trained to identify. Leveraging this, RTD-Guard employs an off-the-shelf RTD discriminator-without fine-tuning-to localize suspicious tokens, masks them, and detects adversarial examples by observing the prediction confidence shift of the victim model before and after intervention. The entire process requires no adversarial data, model tuning, or internal model access, and uses only two black-box queries. Comprehensive experiments on multiple benchmark datasets demonstrate that RTD-Guard effectively detects adversarial texts generated by diverse state-of-the-art attack methods. It surpasses existing detection baselines across multiple metrics, offering a highly efficient, practical, and resource-light defense mechanism-particularly suited for real-world deployment in resource-constrained or privacy-sensitive environments.

RTD-Guard: A Black-Box Textual Adversarial Detection Framework via Replacement Token Detection

Abstract

Textual adversarial attacks pose a serious security threat to Natural Language Processing (NLP) systems by introducing imperceptible perturbations that mislead deep learning models. While adversarial example detection offers a lightweight alternative to robust training, existing methods typically rely on prior knowledge of attacks, white-box access to the victim model, or numerous queries, which severely limits their practical deployment. This paper introduces RTD-Guard, a novel black-box framework for detecting textual adversarial examples. Our key insight is that word-substitution perturbations in adversarial attacks closely resemble the "replaced tokens" that a Replaced Token Detection (RTD) discriminator is pre-trained to identify. Leveraging this, RTD-Guard employs an off-the-shelf RTD discriminator-without fine-tuning-to localize suspicious tokens, masks them, and detects adversarial examples by observing the prediction confidence shift of the victim model before and after intervention. The entire process requires no adversarial data, model tuning, or internal model access, and uses only two black-box queries. Comprehensive experiments on multiple benchmark datasets demonstrate that RTD-Guard effectively detects adversarial texts generated by diverse state-of-the-art attack methods. It surpasses existing detection baselines across multiple metrics, offering a highly efficient, practical, and resource-light defense mechanism-particularly suited for real-world deployment in resource-constrained or privacy-sensitive environments.
Paper Structure (26 sections, 7 equations, 4 figures, 5 tables, 1 algorithm)

This paper contains 26 sections, 7 equations, 4 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Research Objectives
  4. Background
  5. Preliminaries
  6. Problem Statement
  7. The RTD-Guard Framework
  8. Threat Model
  9. Intuition and Motivation: Bridging RTD and Adversarial Perturbations
  10. The RTD-Guard Framework
  11. Experimental Setup
  12. Datasets and Victim Models
  13. Adversarial Attacks
  14. Baseline Detectors
  15. Evaluation Metrics
  16. ...and 11 more sections

Figures (4)

  • Figure 1: Structural symmetry between adversarial attacks and RTD training. Both processes rely on token substitution as the core operation, but differ in their objectives. Adversarial attackers target critical tokens to mislead models, while RTD generators perform random substitutions to train discriminators.
  • Figure 2: Total runtime comparison of different detection methods on the Ag-News/TextFooler split. RTD-Guard is the most efficient.
  • Figure 3: Comparison of detection score distributions on the IMDB/TextFooler subset.
  • Figure 4: Top-k masking performance comparison experiment across datasets and attack methods.