SKDU at De-Factify 4.0: Natural Language Features for AI-Generated Text-Detection

TL;DR

The paper addresses the challenge of distinguishing human-written from AI-generated text by proposing a pipelined detection framework that fuses prompt-based rewriting features (RAIDAR-inspired) with content-based NELA features. Evaluations on the Defactify4.0 dataset cover binary detection and AI-model attribution, with NELA features outperforming RAIDAR and XGBoost emerging as the strongest classifier. Combining the two feature sets yields limited gains due to redundancy, underscoring the primacy of rich content-based attributes for robust detection. The work contributes practical insights for AI-forensics and points toward domain adaptation and feature-invariance strategies to enhance generalizability across evolving LLM families.

Abstract

The rapid advancement of large language models (LLMs) has introduced new challenges in distinguishing human-written text from AI-generated content. In this work, we explored a pipelined approach for AI-generated text detection that includes a feature extraction step (i.e. prompt-based rewriting features inspired by RAIDAR and content-based features derived from the NELA toolkit) followed by a classification module. Comprehensive experiments were conducted on the Defactify4.0 dataset, evaluating two tasks: binary classification to differentiate human-written and AI-generated text, and multi-class classification to identify the specific generative model used to generate the input text. Our findings reveal that NELA features significantly outperform RAIDAR features in both tasks, demonstrating their ability to capture nuanced linguistic, stylistic, and content-based differences. Combining RAIDAR and NELA features provided minimal improvement, highlighting the redundancy introduced by less discriminative features. Among the classifiers tested, XGBoost emerged as the most effective, leveraging the rich feature sets to achieve high accuracy and generalisation.

Figures (2)

• Figure 1: Overview of the Training/Testing pipeline for text classification tasks. The pipeline includes text preprocessing, feature extraction (rewriting-based/NELA features), feature fusion, classifier training, and evaluation.
• Figure 2: Confusion matrix illustrating the classification performance across human-written and AI-generated text classes.