Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Interpretable Text Classification Applied to the Detection of LLM-generated Creative Writing

Minerva Suvanto, Andrea McGlinchey, Mattias Wahde, Peter J Barclay

TL;DR

The paper tackles detecting LLM-generated creative fiction and demonstrates that simple, interpretable linear classifiers can achieve near-perfect accuracy on a held-out test set, vastly outperforming humans who perform at chance. By leveraging only unigram features from carefully prepared text, the authors reveal a constellation of cues—from verb choice to lexical diversity and temporal/varietal markers—that collectively distinguish human from AI-generated writing. Through detailed feature and entropy analyses, they show that LLM rewrites introduce more vocabulary variation and specific rewrites (e.g., synonyms, alternative verbs) that ML models exploit, while maintaining robustness against attempt to evade detection. The work highlights the value of interpretability for trustworthy AI-detection tools in publishing workflows and points to future research across models, genres, and prompts to validate and extend these findings.

Abstract

We consider the problem of distinguishing human-written creative fiction (excerpts from novels) from similar text generated by an LLM. Our results show that, while human observers perform poorly (near chance levels) on this binary classification task, a variety of machine-learning models achieve accuracy in the range 0.93 - 0.98 over a previously unseen test set, even using only short samples and single-token (unigram) features. We therefore employ an inherently interpretable (linear) classifier (with a test accuracy of 0.98), in order to elucidate the underlying reasons for this high accuracy. In our analysis, we identify specific unigram features indicative of LLM-generated text, one of the most important being that the LLM tends to use a larger variety of synonyms, thereby skewing the probability distributions in a manner that is easy to detect for a machine learning classifier, yet very difficult for a human observer. Four additional explanation categories were also identified, namely, temporal drift, Americanisms, foreign language usage, and colloquialisms. As identification of the AI-generated text depends on a constellation of such features, the classification appears robust, and therefore not easy to circumvent by malicious actors intent on misrepresenting AI-generated text as human work.

Interpretable Text Classification Applied to the Detection of LLM-generated Creative Writing

TL;DR

The paper tackles detecting LLM-generated creative fiction and demonstrates that simple, interpretable linear classifiers can achieve near-perfect accuracy on a held-out test set, vastly outperforming humans who perform at chance. By leveraging only unigram features from carefully prepared text, the authors reveal a constellation of cues—from verb choice to lexical diversity and temporal/varietal markers—that collectively distinguish human from AI-generated writing. Through detailed feature and entropy analyses, they show that LLM rewrites introduce more vocabulary variation and specific rewrites (e.g., synonyms, alternative verbs) that ML models exploit, while maintaining robustness against attempt to evade detection. The work highlights the value of interpretability for trustworthy AI-detection tools in publishing workflows and points to future research across models, genres, and prompts to validate and extend these findings.

Abstract

We consider the problem of distinguishing human-written creative fiction (excerpts from novels) from similar text generated by an LLM. Our results show that, while human observers perform poorly (near chance levels) on this binary classification task, a variety of machine-learning models achieve accuracy in the range 0.93 - 0.98 over a previously unseen test set, even using only short samples and single-token (unigram) features. We therefore employ an inherently interpretable (linear) classifier (with a test accuracy of 0.98), in order to elucidate the underlying reasons for this high accuracy. In our analysis, we identify specific unigram features indicative of LLM-generated text, one of the most important being that the LLM tends to use a larger variety of synonyms, thereby skewing the probability distributions in a manner that is easy to detect for a machine learning classifier, yet very difficult for a human observer. Four additional explanation categories were also identified, namely, temporal drift, Americanisms, foreign language usage, and colloquialisms. As identification of the AI-generated text depends on a constellation of such features, the classification appears robust, and therefore not easy to circumvent by malicious actors intent on misrepresenting AI-generated text as human work.
Paper Structure (16 sections, 1 equation, 6 figures, 4 tables)

This paper contains 16 sections, 1 equation, 6 figures, 4 tables.

Table of Contents

  1. INTRODUCTION
  2. BACKGROUND
  3. Text Generation
  4. Related Work
  5. Interpretability
  6. METHODOLOGY
  7. Data Sets
  8. Text Classifiers and Features
  9. Artefacts of the Generation Process
  10. RESULTS
  11. Classification by Human Evaluators
  12. Classification Using ML Methods
  13. Analysis and Discussion
  14. Feature Analysis
  15. Entropy Analysis
  16. ...and 1 more sections

Figures (6)

  • Figure 1: Analysis of linguistic features on our DET12 training data set, showing the 10 features that exhibited the smallest difference between the two classes. The horizontal axis measures the (relative) frequency, i.e., the number of instances per million tokens.
  • Figure 2: Analysis of linguistic features on our DET12 training data set. The left panel shows the 10 features that had the largest frequency increase in Class 1 (when compared to Class 0), while the right panel shows the 10 features with the largest decrease in Class 1. The horizontal axis measures the (relative) frequency, i.e., the number of instances per million tokens. The different feature categories are explained in https://browndw.github.io/pybiber/feature-categories.html.
  • Figure 3: Visualisation of the classification result using the interpretable classifier, where one can inspect the precise impact of each unigram feature on the classification result. Yellow is used to highlight negative weights (Class 0, human-written) and purple for positive weights (Class 1, LLM-generated). The text pair is a sample from the test set. Stronger colour saturations indicate higher weight magnitudes of the features. Explanation codes are shown above features that were part of the manual annotation procedure.
  • Figure 4: Trend for the use of exited from 1910 to 2020; data obtained from Google $n-$gram viewer; https://books.google.com/ngrams/.
  • Figure 5: Drop in test set accuracy when features are removed from consideration. Top (blue) curve: Removal of features in falling order of their frequency; Bottom (orange) curve: Removal only of features with $|w| \ge 0.1$ (again sorted in falling order of frequency). See the main text for a more complete description.
  • ...and 1 more figures