StyleDecipher: Robust and Explainable Detection of LLM-Generated Texts with Stylistic Analysis

TL;DR

The paper tackles robust detection of LLM-generated text across diverse domains without relying on model internals. StyleDecipher encodes each text into discrete and continuous style features by comparing the original text $x$ with semantically preserved rewrites $\\hat{x}$, using metrics like $v_N(x)$, $v_{\\text{edit}}(x)$ and $v_C(x)$, then fuses them into a classifier. It demonstrates strong in-domain and cross-domain performance, plus resilience to adversarial perturbations and data mixing, including RAID benchmarks; the approach yields explainable attributions via visualization and quantitative metrics. The work provides a practical, modular framework that can be plugged with various embeddings and maintains transparency, enabling reliable detection without access to model internals.

Abstract

With the increasing integration of large language models (LLMs) into open-domain writing, detecting machine-generated text has become a critical task for ensuring content authenticity and trust. Existing approaches rely on statistical discrepancies or model-specific heuristics to distinguish between LLM-generated and human-written text. However, these methods struggle in real-world scenarios due to limited generalization, vulnerability to paraphrasing, and lack of explainability, particularly when facing stylistic diversity or hybrid human-AI authorship. In this work, we propose StyleDecipher, a robust and explainable detection framework that revisits LLM-generated text detection using combined feature extractors to quantify stylistic differences. By jointly modeling discrete stylistic indicators and continuous stylistic representations derived from semantic embeddings, StyleDecipher captures distinctive style-level divergences between human and LLM outputs within a unified representation space. This framework enables accurate, explainable, and domain-agnostic detection without requiring access to model internals or labeled segments. Extensive experiments across five diverse domains, including news, code, essays, reviews, and academic abstracts, demonstrate that StyleDecipher consistently achieves state-of-the-art in-domain accuracy. Moreover, in cross-domain evaluations, it surpasses existing baselines by up to 36.30%, while maintaining robustness against adversarial perturbations and mixed human-AI content. Further qualitative and quantitative analysis confirms that stylistic signals provide explainable evidence for distinguishing machine-generated text. Our source code can be accessed at https://github.com/SiyuanLi00/StyleDecipher.

Figures (8)

• Figure 1: 2D UMAP projection of stylistic features for LLM-generated texts and human-written texts on the Yelp Review dataset. Each point represents an embedded paragraph ($\leq$ 64 tokens) encoded with combined stylistic features. LLM-generated texts exhibit consistent stylistic patterns, forming distinguishable clusters from human-written texts.
• Figure 2: Overview of the proposed StyleDecipher. ① Given an input text $x$, a rewrite LLM model (GPT-3.5-turbo) generates a set of rewritten copies, denoted as $x^{\prime}_i$. ② The original text $x$ and its corresponding rewritten copies are then channeled into our encoder. This extractor is designed to perform both discrete feature and continuous feature extraction. Subsequently, the resulting feature vectors of these texts are compared using a similarity metric. This comparative analysis yields a derived set of vectors $v$, which are specifically engineered for subsequent classification. ③ Processing this vector $v$ by a binary classifier to determine the source of the given text $x$.
• Figure 3: Cross-domain generalization F1 score on ID-OOD splits. Each subfigure represents the results on a source training domain, where each bar shows the result on the target evaluation domain. The last subfigure denotes the average F1 score on out-of-domain targets.
• Figure 4: Evaluation of detection performance degradation under adversarial perturbations: Comparative analysis of original AUROC, AUROC after perturbation, and relative drop rate across News, HumanEval, Code, and Yelp Review datasets.
• Figure 5: Evaluation of detection performance degradation under adversarial mixed text: Comparative analysis of original AUROC, AUROC after mixing, and relative performance drop across News, HumanEval, Code, and Yelp Review datasets.