Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text

Jenna Russell, Marzena Karpinska, Mohit Iyyer

TL;DR

This work investigates human ability to detect AI-generated text produced by modern large language models, evaluating whether experts who frequently use LLMs for writing tasks can outperform automated detectors. Through five within-subject experiments on 300 English nonfiction articles, the study shows that expert annotators achieve near-perfect detection, with a majority vote misclassifying only 1 article out of 300, and often exceed many commercial detectors—occasionally matching Pangram’s performance. Analyses of expert explanations reveal a mix of lexical cues (AI vocabulary) and higher-level signals (formality, originality, tone) that remain informative even under paraphrasing and humanization attacks, suggesting humans possess robust and explainable detection capabilities. The paper also demonstrates that while LLM-driven prompt detectors can approach some performance, they generally lag behind expert humans, particularly under adversarial manipulation, highlighting the value of human-in-the-loop efforts and potential collaboration between humans and automated detectors for reliable AI-text detection in high-stakes contexts.

Abstract

In this paper, we study how well humans can detect text generated by commercial LLMs (GPT-4o, Claude, o1). We hire annotators to read 300 non-fiction English articles, label them as either human-written or AI-generated, and provide paragraph-length explanations for their decisions. Our experiments show that annotators who frequently use LLMs for writing tasks excel at detecting AI-generated text, even without any specialized training or feedback. In fact, the majority vote among five such "expert" annotators misclassifies only 1 of 300 articles, significantly outperforming most commercial and open-source detectors we evaluated even in the presence of evasion tactics like paraphrasing and humanization. Qualitative analysis of the experts' free-form explanations shows that while they rely heavily on specific lexical clues ('AI vocabulary'), they also pick up on more complex phenomena within the text (e.g., formality, originality, clarity) that are challenging to assess for automatic detectors. We release our annotated dataset and code to spur future research into both human and automated detection of AI-generated text.

People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text

TL;DR

This work investigates human ability to detect AI-generated text produced by modern large language models, evaluating whether experts who frequently use LLMs for writing tasks can outperform automated detectors. Through five within-subject experiments on 300 English nonfiction articles, the study shows that expert annotators achieve near-perfect detection, with a majority vote misclassifying only 1 article out of 300, and often exceed many commercial detectors—occasionally matching Pangram’s performance. Analyses of expert explanations reveal a mix of lexical cues (AI vocabulary) and higher-level signals (formality, originality, tone) that remain informative even under paraphrasing and humanization attacks, suggesting humans possess robust and explainable detection capabilities. The paper also demonstrates that while LLM-driven prompt detectors can approach some performance, they generally lag behind expert humans, particularly under adversarial manipulation, highlighting the value of human-in-the-loop efforts and potential collaboration between humans and automated detectors for reliable AI-text detection in high-stakes contexts.

Abstract

In this paper, we study how well humans can detect text generated by commercial LLMs (GPT-4o, Claude, o1). We hire annotators to read 300 non-fiction English articles, label them as either human-written or AI-generated, and provide paragraph-length explanations for their decisions. Our experiments show that annotators who frequently use LLMs for writing tasks excel at detecting AI-generated text, even without any specialized training or feedback. In fact, the majority vote among five such "expert" annotators misclassifies only 1 of 300 articles, significantly outperforming most commercial and open-source detectors we evaluated even in the presence of evasion tactics like paraphrasing and humanization. Qualitative analysis of the experts' free-form explanations shows that while they rely heavily on specific lexical clues ('AI vocabulary'), they also pick up on more complex phenomena within the text (e.g., formality, originality, clarity) that are challenging to assess for automatic detectors. We release our annotated dataset and code to spur future research into both human and automated detection of AI-generated text.
Paper Structure (71 sections, 14 figures, 23 tables)

This paper contains 71 sections, 14 figures, 23 tables.

Table of Contents

  1. Introduction
  2. Experience matters:
  3. Human experts outperform automatic detectors:
  4. What do expert annotators focus on?
  5. Can LLMs mimic human detectors?
  6. How good are humans at detecting AI-generated text?
  7. Task setup:
  8. Article selection:
  9. Generating paired AI articles:
  10. Annotator details:
  11. Justification of sample size:
  12. Experiment 1: What kinds of annotators can reliably detect AI-generated articles?
  13. Annotators with limited LLM experience are unreliable detectors:
  14. Annotators who frequently use LLMs for writing tasks are highly accurate:
  15. What do expert annotators see that nonexperts don't?
  16. ...and 56 more sections

Figures (14)

  • Figure 1: A human expert's annotations of an article generated by OpenAI's o1-Pro with humanization. The expert provides a judgment on whether the text is written by a human or AI, a confidence score, and an explanation (including both free-form text and highlighted spans) of their decision.
  • Figure 2: Expert confidence in their decisions drops when judging humanized articles generated by o1-Pro.
  • Figure 3: A heatmap displaying the frequency with which annotators mentioned specific categories in their explanations when they were correct. Interestingly, vocabulary becomes a less frequent clue for o1-Pro-generated articles, especially with humanization. A heat map of corresponding incorrect explanation is displayed in \ref{['fig:heatmaps_incorrect']}. Details of each category can be found in \ref{['tab:explanation_category_definitions']}.
  • Figure 4: Guidelines provided to the annotators for the annotation task. The annotators were also provided additional examples and guidance during the data collection process.
  • Figure 5: Consent form which the annotators were asked to sign via GoogleForms before collecting the data.
  • ...and 9 more figures