Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DependencyAI: Detecting AI Generated Text through Dependency Parsing

Sara Ahmed, Tracy Hammond

TL;DR

The results demonstrate that dependency relations alone provide a robust signal for AI-generated text detection, establishing DependencyAI as a strong linguistically grounded, interpretable, and non-neural network baseline.

Abstract

As large language models (LLMs) become increasingly prevalent, reliable methods for detecting AI-generated text are critical for mitigating potential risks. We introduce DependencyAI, a simple and interpretable approach for detecting AI-generated text using only the labels of linguistic dependency relations. Our method achieves competitive performance across monolingual, multi-generator, and multilingual settings. To increase interpretability, we analyze feature importance to reveal syntactic structures that distinguish AI-generated from human-written text. We also observe a systematic overprediction of certain models on unseen domains, suggesting that generator-specific writing styles may affect cross-domain generalization. Overall, our results demonstrate that dependency relations alone provide a robust signal for AI-generated text detection, establishing DependencyAI as a strong linguistically grounded, interpretable, and non-neural network baseline.

DependencyAI: Detecting AI Generated Text through Dependency Parsing

TL;DR

The results demonstrate that dependency relations alone provide a robust signal for AI-generated text detection, establishing DependencyAI as a strong linguistically grounded, interpretable, and non-neural network baseline.

Abstract

As large language models (LLMs) become increasingly prevalent, reliable methods for detecting AI-generated text are critical for mitigating potential risks. We introduce DependencyAI, a simple and interpretable approach for detecting AI-generated text using only the labels of linguistic dependency relations. Our method achieves competitive performance across monolingual, multi-generator, and multilingual settings. To increase interpretability, we analyze feature importance to reveal syntactic structures that distinguish AI-generated from human-written text. We also observe a systematic overprediction of certain models on unseen domains, suggesting that generator-specific writing styles may affect cross-domain generalization. Overall, our results demonstrate that dependency relations alone provide a robust signal for AI-generated text detection, establishing DependencyAI as a strong linguistically grounded, interpretable, and non-neural network baseline.
Paper Structure (14 sections, 3 figures, 2 tables)

This paper contains 14 sections, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Experimental Setup
  5. Dataset
  6. Tasks
  7. Baselines
  8. Results
  9. Multi-way Detection
  10. Multilingual Detection
  11. Effect of Dependency Relation N-grams
  12. Feature Importance
  13. Conclusion
  14. Limitations

Figures (3)

  • Figure 1: Distribution of prediction errors by predicted class for the multi-way detection task on domains excluded from training. Percentages indicate the proportion of errors attributed to each predicted class.
  • Figure 2: Dependency relation n-gram range vs. accuracy by language
  • Figure 3: Top-5 feature importance by gain across languages. Each subplot corresponds to a language (Chinese, English, German, Italian, Russian) and the aggregated dataset (All), showing the five dependency features with the highest gain values for each.