Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Red Teaming Language Model Detectors with Language Models

Zhouxing Shi, Yihan Wang, Fan Yin, Xiangning Chen, Kai-Wei Chang, Cho-Jui Hsieh

TL;DR

This work reveals that existing LLM-generated text detectors, including DetectGPT, watermarking, and classifier based systems, are vulnerable to adversarial attacks that preserve text plausibility. It introduces two red-teaming strategies: word substitutions generated by a protected attacker LLM and instructional prompts that steer generation away from detectable distributions, both under black-box constraints. Across GPT-2-XL, LLaMA-65B, and ChatGPT, the attacks substantially degrade detector performance measured by AUROC, DR, and ASR, while human judgments indicate only modest quality loss. The findings highlight an urgent need for robust detectors, including hybrid approaches that couple watermarking with likelihood-based checks to improve resilience against evolving evasion tactics.

Abstract

The prevalence and strong capability of large language models (LLMs) present significant safety and ethical risks if exploited by malicious users. To prevent the potentially deceptive usage of LLMs, recent works have proposed algorithms to detect LLM-generated text and protect LLMs. In this paper, we investigate the robustness and reliability of these LLM detectors under adversarial attacks. We study two types of attack strategies: 1) replacing certain words in an LLM's output with their synonyms given the context; 2) automatically searching for an instructional prompt to alter the writing style of the generation. In both strategies, we leverage an auxiliary LLM to generate the word replacements or the instructional prompt. Different from previous works, we consider a challenging setting where the auxiliary LLM can also be protected by a detector. Experiments reveal that our attacks effectively compromise the performance of all detectors in the study with plausible generations, underscoring the urgent need to improve the robustness of LLM-generated text detection systems.

Red Teaming Language Model Detectors with Language Models

TL;DR

This work reveals that existing LLM-generated text detectors, including DetectGPT, watermarking, and classifier based systems, are vulnerable to adversarial attacks that preserve text plausibility. It introduces two red-teaming strategies: word substitutions generated by a protected attacker LLM and instructional prompts that steer generation away from detectable distributions, both under black-box constraints. Across GPT-2-XL, LLaMA-65B, and ChatGPT, the attacks substantially degrade detector performance measured by AUROC, DR, and ASR, while human judgments indicate only modest quality loss. The findings highlight an urgent need for robust detectors, including hybrid approaches that couple watermarking with likelihood-based checks to improve resilience against evolving evasion tactics.

Abstract

The prevalence and strong capability of large language models (LLMs) present significant safety and ethical risks if exploited by malicious users. To prevent the potentially deceptive usage of LLMs, recent works have proposed algorithms to detect LLM-generated text and protect LLMs. In this paper, we investigate the robustness and reliability of these LLM detectors under adversarial attacks. We study two types of attack strategies: 1) replacing certain words in an LLM's output with their synonyms given the context; 2) automatically searching for an instructional prompt to alter the writing style of the generation. In both strategies, we leverage an auxiliary LLM to generate the word replacements or the instructional prompt. Different from previous works, we consider a challenging setting where the auxiliary LLM can also be protected by a detector. Experiments reveal that our attacks effectively compromise the performance of all detectors in the study with plausible generations, underscoring the urgent need to improve the robustness of LLM-generated text detection systems.
Paper Structure (32 sections, 3 equations, 6 figures, 14 tables)

This paper contains 32 sections, 3 equations, 6 figures, 14 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Detectors for AI-generated text.
  4. Methods for red-teaming detectors.
  5. Adversarial examples in NLP.
  6. Safety of large language models.
  7. Settings and Overview
  8. Attack with Word Substitutions
  9. General attack objective.
  10. Generating Word Substitution Candidates
  11. Query-based Word Substitutions
  12. Query-free Word Substitutions
  13. Attack by Instructional Prompts
  14. Searching for ${\mathbf{X}}_p$.
  15. Experiments
  16. ...and 17 more sections

Figures (6)

  • Figure 1: ROC plot of OpenAI AI Text Classifier under different attack methods. We show the ROC plot on the ELI5 dataset in \ref{['tab:openai_detector']}.
  • Figure 2: ROC plot of DetectGPT detectors under different attack methods. We show the ROC plot on the ELI5 dataset in \ref{['tab:detectgpt_exp']} for the GPT2-XL model.
  • Figure 3: ROC plot of watermarking detectors under different attack methods. We show the ROC plot on the ELI5 dataset in \ref{['tab:watermark_exp']} for the GPT2-XL model with $\delta=1.5$.
  • Figure 4: ROC plot of watermarking detectors under different attack methods. We show the ROC plot on the ELI5 dataset in \ref{['tab:watermark_exp']} for the LLaMA-65B model with $\delta=1.5$.
  • Figure 5: ROC plot of watermarking detectors under query-free attacks with different replacement ratios. We show the ROC plot on the ELI5 dataset for the GPT2-XL model with $\delta=1.5$.
  • ...and 1 more figures