Iron Sharpens Iron: Defending Against Attacks in Machine-Generated Text Detection with Adversarial Training

TL;DR

This work addresses the vulnerability of machine-generated text detectors to perturbations and adversarial attacks. It introduces GREATER, a co-trained framework with GREATER-A (adversary) and GREATER-D (detector) that operates in a black-box setting by leveraging a surrogate model to craft targeted token-level perturbations and a greedy refinement process. Through adversarial training, GREATER-D learns robust features that generalize across attacks and backbones, achieving state-of-the-art defense performance while GREATER-A delivers highly effective, query-efficient attacks. The study demonstrates substantial gains in robustness and attack efficiency, with multilingual and cross-model validations, while also discussing practical limitations and ethical considerations for deploying such defenses.

Abstract

Machine-generated Text (MGT) detection is crucial for regulating and attributing online texts. While the existing MGT detectors achieve strong performance, they remain vulnerable to simple perturbations and adversarial attacks. To build an effective defense against malicious perturbations, we view MGT detection from a threat modeling perspective, that is, analyzing the model's vulnerability from an adversary's point of view and exploring effective mitigations. To this end, we introduce an adversarial framework for training a robust MGT detector, named GREedy Adversary PromoTed DefendER (GREATER). The GREATER consists of two key components: an adversary GREATER-A and a detector GREATER-D. The GREATER-D learns to defend against the adversarial attack from GREATER-A and generalizes the defense to other attacks. GREATER-A identifies and perturbs the critical tokens in embedding space, along with greedy search and pruning to generate stealthy and disruptive adversarial examples. Besides, we update the GREATER-A and GREATER-D synchronously, encouraging the GREATER-D to generalize its defense to different attacks and varying attack intensities. Our experimental results across 10 text perturbation strategies and 6 adversarial attacks show that our GREATER-D reduces the Attack Success Rate (ASR) by 0.67% compared with SOTA defense methods while our GREATER-A is demonstrated to be more effective and efficient than SOTA attack approaches. Codes and dataset are available in https://github.com/Liyuuuu111/GREATER.

Figures (4)

• Figure 1: Performance drop of different MGT detectors and defense methods under text perturbation.
• Figure 2: Pipeline of Greater. The adversary identifies important tokens in the original MGT and generates candidates for important tokens (§\ref{['AEG']}). The adversary conducts and refines the attack by greedy search and pruning (§\ref{['AEE']}). The final adversarial examples are feeded to the target detector (§\ref{['DA']}) and participate in the adversarial training process (§\ref{['AT']}).
• Figure 3: Impact of attack strength in Greater. We normalize the attack strength for better visualization of the results.
• Figure 4: Defense performance under attack with different strengths. A lower ASR(%) indicates better defensive performance. A larger character edit distance indicates greater attack intensity in the Mixed Edit Attack. A lower BERT score corresponds to stronger attacks in the Paraphrasing Attack and Code-Switching Attack. A higher obfuscation ratio reflects greater intensity in the Human Obfuscation Attack. Similarly, for PWWS, BERTAttack, TextFooler, and A2T, a larger maximum query count signifies a stronger attack.

Theorems & Definitions (9)

• Definition 1
• Definition 2
• Theorem 1
• proof
• Theorem 2
• proof
• Definition 3
• Theorem 3
• proof