A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions

TL;DR

This survey critically inventories the landscape of LLM-generated text detection, outlining the problem, datasets, and taxonomy of detector methods. It categorizes detectors into watermarking, statistics-based, neural-based, and human-assisted approaches, and discusses their robustness to out-of-distribution data and adversarial attacks. The authors evaluate current data resources and benchmarks, highlight gaps in evaluation frameworks, and underscore the need for robust, real-world-aligned detectors. They conclude with forward-looking directions, including attack-aware detector design, zero-shot efficacy, and misinformation-discrimination capabilities to advance responsible AI governance.

Abstract

The powerful ability to understand, follow, and generate complex language emerging from large language models (LLMs) makes LLM-generated text flood many areas of our daily lives at an incredible speed and is widely accepted by humans. As LLMs continue to expand, there is an imperative need to develop detectors that can detect LLM-generated text. This is crucial to mitigate potential misuse of LLMs and safeguard realms like artistic expression and social networks from harmful influence of LLM-generated content. The LLM-generated text detection aims to discern if a piece of text was produced by an LLM, which is essentially a binary classification task. The detector techniques have witnessed notable advancements recently, propelled by innovations in watermarking techniques, statistics-based detectors, neural-base detectors, and human-assisted methods. In this survey, we collate recent research breakthroughs in this area and underscore the pressing need to bolster detector research. We also delve into prevalent datasets, elucidating their limitations and developmental requirements. Furthermore, we analyze various LLM-generated text detection paradigms, shedding light on challenges like out-of-distribution problems, potential attacks, real-world data issues and the lack of effective evaluation framework. Conclusively, we highlight interesting directions for future research in LLM-generated text detection to advance the implementation of responsible artificial intelligence (AI). Our aim with this survey is to provide a clear and comprehensive introduction for newcomers while also offering seasoned researchers a valuable update in the field of LLM-generated text detection. The useful resources are publicly available at: https://github.com/NLP2CT/LLM-generated-Text-Detection.

Figures (4)

• Figure 1: Toy picture of LLM-generated text detection task. This task is a binary classification task that detects whether the provided text is generated by LLMs or written by humans.
• Figure 2: The most critical reasons why LLM-generated text detection is needed urgently. We discussed it from five perspectives: Regulation, Users, Developments, Science, and Human Society.
• Figure 3: The distribution by year of the last 5 years of literature obtained from the screening is plotted. The number of published articles obtain significant attention in 2023.
• Figure 4: Classification of LLM-generated text detectors with corresponding diagrams and paper lists. We categorize the detectors into watermarking technology, statistics-based detectors, neural-based detectors, and human-assisted methods. In the diagrams, HWT represents Human-Written Text and LGT represents LLM-Generated Text. We use the orange lines to highlight the source of the detector's detection capability, and the green lines to describe the detection process.