LLM-DetectAIve: a Tool for Fine-Grained Machine-Generated Text Detection

TL;DR

This work targets the rising challenge of differentiating human-written and machine-generated text by introducing LLM-DetectAIve, a four-way fine-grained classification framework. It extends the M4GT-Bench dataset across four categories and multiple domains, builds detectors using RoBERTa, DistilBERT, and DeBERTa, and demonstrates that domain-specific, universal, and DANN-augmented detectors can achieve strong performance, with DeBERTa often outperforming alternatives. A public web demo provides automatic detection and a human-detection playground, highlighting practical deployment via Hugging Face Spaces and Gradio. The study also analyzes generalization across unseen domains and discusses limitations, biases, and future directions, including multilingual expansion and additional categories for more robust truth-tracking in educational and forensic contexts.

Abstract

The ease of access to large language models (LLMs) has enabled a widespread of machine-generated texts, and now it is often hard to tell whether a piece of text was human-written or machine-generated. This raises concerns about potential misuse, particularly within educational and academic domains. Thus, it is important to develop practical systems that can automate the process. Here, we present one such system, LLM-DetectAIve, designed for fine-grained detection. Unlike most previous work on machine-generated text detection, which focused on binary classification, LLM-DetectAIve supports four categories: (i) human-written, (ii) machine-generated, (iii) machine-written, then machine-humanized, and (iv) human-written, then machine-polished. Category (iii) aims to detect attempts to obfuscate the fact that a text was machine-generated, while category (iv) looks for cases where the LLM was used to polish a human-written text, which is typically acceptable in academic writing, but not in education. Our experiments show that LLM-DetectAIve can effectively identify the above four categories, which makes it a potentially useful tool in education, academia, and other domains. LLM-DetectAIve is publicly accessible at https://github.com/mbzuai-nlp/LLM-DetectAIve. The video describing our system is available at https://youtu.be/E8eT_bE7k8c.

Figures (2)

• Figure 1: LLM-DetectAIve interface: automatic text detection (top) and human detector playground (bottom).
• Figure 2: Domain-specific confusion matrix for RoBERTa on arXiv (top) and on OUTFOX (bottom).