Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MASH: Evading Black-Box AI-Generated Text Detectors via Style Humanization

Yongtong Gu, Songze Li, Xia Hu

TL;DR

MASH reframes AI-generated text detection as a style transfer problem and presents a four-stage black-box attack pipeline that few-shot tunes AI text to human-like style while preserving quality. Through Data Construction, Style-Injection SFT, DPO Alignment, and Inference-Time Refinement, it achieves high attack success (average ASR ≈ 92%) across six domains and five detectors, outperforming 11 baselines. The approach demonstrates strong robustness and transferability, including to commercial detectors, while also revealing a notable defense-robustness trade-off when detectors are adversarially trained. Overall, MASH serves as an effective red-teaming tool and a basis for strengthening detector systems against style-based evasion.

Abstract

The increasing misuse of AI-generated texts (AIGT) has motivated the rapid development of AIGT detection methods. However, the reliability of these detectors remains fragile against adversarial evasions. Existing attack strategies often rely on white-box assumptions or demand prohibitively high computational and interaction costs, rendering them ineffective under practical black-box scenarios. In this paper, we propose Multi-stage Alignment for Style Humanization (MASH), a novel framework that evades black-box detectors based on style transfer. MASH sequentially employs style-injection supervised fine-tuning, direct preference optimization, and inference-time refinement to shape the distributions of AI-generated texts to resemble those of human-written texts. Experiments across 6 datasets and 5 detectors demonstrate the superior performance of MASH over 11 baseline evaders. Specifically, MASH achieves an average Attack Success Rate (ASR) of 92%, surpassing the strongest baselines by an average of 24%, while maintaining superior linguistic quality.

MASH: Evading Black-Box AI-Generated Text Detectors via Style Humanization

TL;DR

MASH reframes AI-generated text detection as a style transfer problem and presents a four-stage black-box attack pipeline that few-shot tunes AI text to human-like style while preserving quality. Through Data Construction, Style-Injection SFT, DPO Alignment, and Inference-Time Refinement, it achieves high attack success (average ASR ≈ 92%) across six domains and five detectors, outperforming 11 baselines. The approach demonstrates strong robustness and transferability, including to commercial detectors, while also revealing a notable defense-robustness trade-off when detectors are adversarially trained. Overall, MASH serves as an effective red-teaming tool and a basis for strengthening detector systems against style-based evasion.

Abstract

The increasing misuse of AI-generated texts (AIGT) has motivated the rapid development of AIGT detection methods. However, the reliability of these detectors remains fragile against adversarial evasions. Existing attack strategies often rely on white-box assumptions or demand prohibitively high computational and interaction costs, rendering them ineffective under practical black-box scenarios. In this paper, we propose Multi-stage Alignment for Style Humanization (MASH), a novel framework that evades black-box detectors based on style transfer. MASH sequentially employs style-injection supervised fine-tuning, direct preference optimization, and inference-time refinement to shape the distributions of AI-generated texts to resemble those of human-written texts. Experiments across 6 datasets and 5 detectors demonstrate the superior performance of MASH over 11 baseline evaders. Specifically, MASH achieves an average Attack Success Rate (ASR) of 92%, surpassing the strongest baselines by an average of 24%, while maintaining superior linguistic quality.
Paper Structure (26 sections, 3 theorems, 15 equations, 17 figures, 10 tables)

This paper contains 26 sections, 3 theorems, 15 equations, 17 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. AI-generated Text Detection
  4. Detection Evasion Methods
  5. Text Style Transfer
  6. Methodology
  7. Problem Formulation
  8. MASH Methodology
  9. Experiments
  10. Experimental Setup
  11. Performance Evaluation
  12. Ablation Study
  13. Universality and Transferability Analysis
  14. Defense via Adversarial Training
  15. Conclusion
  16. ...and 11 more sections

Key Result

Theorem 1

Let $\pi_{\text{ref}}$ be the reference policy and $\pi^*$ be the optimal policy minimizing the DPO loss. The generated distribution $\pi^*$ aligns with the human-like regions defined by the detector $D$ (i.e., regions where $D(\mathbf{y}) \to 0$).

Figures (17)

  • Figure 1: Illustration of an evasion attack against AIGT detectors in a black-box setting.
  • Figure 2: Overview of the proposed MASH framework. It comprises four stages: (1) Data Construction to synthesize parallel data; (2) Style-Injection SFT for supervised initialization; (3) DPO Alignment to optimize against detector boundaries; and (4) Inference-Time Adversarial Refinement to guarantee final text quality.
  • Figure 3: ROC curves comparing the evasion effectiveness of MASH against baseline attacks on the RoBERTa detector across the WP, STEM, and Social domains. Note that a lower TPR corresponds to a higher ASR.
  • Figure 4: Ablation study of MASH components, averaged across three detectors (RoBERTa, Binoculars, SCRN) and six datasets.
  • Figure 5: Impact of data source selection.
  • ...and 12 more figures

Theorems & Definitions (6)

  • Theorem 1: Optimality of Evasion
  • proof
  • Proposition 1: Optimization Consistency & Gradient Efficiency
  • proof
  • Theorem 2: Support Constraint
  • proof