The Best Defense is Attack: Repairing Semantics in Textual Adversarial Examples

TL;DR

This work targets textual adversarial robustness by identifying the semantic drift caused by adversaries and proposes Rapid, a two-phase defense that couples an in-model adversarial detector with a reactive perturbation-based repair mechanism. Phase 1 jointly trains a victim classifier and a detector using multi-attack adversaries to enable targeted defense, formalized by a composite loss that balances classification, detection, and adversarial training. Phase 2 applies detection, then repairs adversaries through perturbation defocusing and pseudo-semantic similarity filtering to preserve natural semantics while minimizing edits. Experiments on SST2, Amazon, AGNews, and Yahoo! demonstrate high detection accuracy, substantial repair rates (up to 99%), and notable efficiency gains over prior methods, with robustness to unseen attacks and different victim-model backbones like BERT and DeBERTa. The work contributes a reproducible benchmarking platform and highlights the practical potential of semantics-focused defenses in real-world NLP tasks.

Abstract

Recent studies have revealed the vulnerability of pre-trained language models to adversarial attacks. Existing adversarial defense techniques attempt to reconstruct adversarial examples within feature or text spaces. However, these methods struggle to effectively repair the semantics in adversarial examples, resulting in unsatisfactory performance and limiting their practical utility. To repair the semantics in adversarial examples, we introduce a novel approach named Reactive Perturbation Defocusing (Rapid). Rapid employs an adversarial detector to identify fake labels of adversarial examples and leverage adversarial attackers to repair the semantics in adversarial examples. Our extensive experimental results conducted on four public datasets, convincingly demonstrate the effectiveness of Rapid in various adversarial attack scenarios. To address the problem of defense performance validation in previous works, we provide a demonstration of adversarial detection and repair based on our work, which can be easily evaluated at https://tinyurl.com/22ercuf8.

Figures (5)

• Figure 1: Box plots of the cosine similarity between the adversary--natural example pairs (marked in red) and the repaired adversary--natural example pairs obtained by Rapid versus RS&V. The cosine similarity is evaluated based on the features extracted by the victim models of Rapid and RS&V, respectively. The larger the cosine similarity, the more similar the corresponding example pair. It is observed that the victim model cannot discern the semantic differences between the adversaries and the repaired adversaries produced by RS&V, whereas Rapid can precisely differentiate between adversaries and natural examples. Conversely, when using Rapid, the repaired adversaries regain their semantic alignment with the natural examples.
• Figure 2: A pedagogical example of Rapid in sentiment analysis. The original word in this example is exploration. Perturbation defocusing repairs the adversary by injecting perturbations (interesting) to distract the objective model from the malicious perturbation (i.e., investigation). Rapid only implements defense on the pre-detected adversary.
• Figure 3: The overall architecture and workflow of Rapid.
• Figure 4: Box plots of semantic cosine similarity score distributions on multi-categorial datasets. Similar to fig:intro, Rapid is more competent to repair semantics according to the feature similarity score distributions.
• Figure 5: The demo examples of adversarial detection and defense built on Rapid for defending against multi-attacks. The comparisons between natural and repaired examples are available based on the "difflib" library. The "$+$" and "$-$" in the colored boxes indicate letters addition and deletion compared to the natural examples. It is observed that Rapid only injects only one perturbation to repair the adversarial example, i.e., changing "screw" to "bang" in the adversarial example.

Theorems & Definitions (2)

• Remark 1
• Remark 2