Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DETree: DEtecting Human-AI Collaborative Texts via Tree-Structured Hierarchical Representation Learning

Yongxin He, Shan Zhang, Yixuan Cao, Lei Ma, Ping Luo

TL;DR

This work tackles the challenge of detecting AI involvement in text when collaboration between humans and multiple AI agents produces hybrid content. It introduces DETree, a representation-learning framework built on a Hierarchical Affinity Tree (HAT) and a Tree-Structured Contrastive Loss (TSCL) to capture latent relational structures among generation sources, paired with RealBench to simulate realistic hybrid-text distributions. DETree achieves state-of-the-art performance and robust generalization under out-of-distribution and few-shot conditions, aided by K-means based database compression and retrieval-based adaptation. The approach provides interpretable insights into how different AI sources cluster and interact, with practical implications for improving transparency and accountability in AI-assisted writing. The combination of hierarchical source modeling, structured contrastive learning, and retrieval-based adaptation offers a promising direction for detection systems facing rapidly evolving LLMs and hybrid authorship scenarios.

Abstract

Detecting AI-involved text is essential for combating misinformation, plagiarism, and academic misconduct. However, AI text generation includes diverse collaborative processes (AI-written text edited by humans, human-written text edited by AI, and AI-generated text refined by other AI), where various or even new LLMs could be involved. Texts generated through these varied processes exhibit complex characteristics, presenting significant challenges for detection. Current methods model these processes rather crudely, primarily employing binary classification (purely human vs. AI-involved) or multi-classification (treating human-AI collaboration as a new class). We observe that representations of texts generated through different processes exhibit inherent clustering relationships. Therefore, we propose DETree, a novel approach that models the relationships among different processes as a Hierarchical Affinity Tree structure, and introduces a specialized loss function that aligns text representations with this tree. To facilitate this learning, we developed RealBench, a comprehensive benchmark dataset that automatically incorporates a wide spectrum of hybrid texts produced through various human-AI collaboration processes. Our method improves performance in hybrid text detection tasks and significantly enhances robustness and generalization in out-of-distribution scenarios, particularly in few-shot learning conditions, further demonstrating the promise of training-based approaches in OOD settings. Our code and dataset are available at https://github.com/heyongxin233/DETree.

DETree: DEtecting Human-AI Collaborative Texts via Tree-Structured Hierarchical Representation Learning

TL;DR

This work tackles the challenge of detecting AI involvement in text when collaboration between humans and multiple AI agents produces hybrid content. It introduces DETree, a representation-learning framework built on a Hierarchical Affinity Tree (HAT) and a Tree-Structured Contrastive Loss (TSCL) to capture latent relational structures among generation sources, paired with RealBench to simulate realistic hybrid-text distributions. DETree achieves state-of-the-art performance and robust generalization under out-of-distribution and few-shot conditions, aided by K-means based database compression and retrieval-based adaptation. The approach provides interpretable insights into how different AI sources cluster and interact, with practical implications for improving transparency and accountability in AI-assisted writing. The combination of hierarchical source modeling, structured contrastive learning, and retrieval-based adaptation offers a promising direction for detection systems facing rapidly evolving LLMs and hybrid authorship scenarios.

Abstract

Detecting AI-involved text is essential for combating misinformation, plagiarism, and academic misconduct. However, AI text generation includes diverse collaborative processes (AI-written text edited by humans, human-written text edited by AI, and AI-generated text refined by other AI), where various or even new LLMs could be involved. Texts generated through these varied processes exhibit complex characteristics, presenting significant challenges for detection. Current methods model these processes rather crudely, primarily employing binary classification (purely human vs. AI-involved) or multi-classification (treating human-AI collaboration as a new class). We observe that representations of texts generated through different processes exhibit inherent clustering relationships. Therefore, we propose DETree, a novel approach that models the relationships among different processes as a Hierarchical Affinity Tree structure, and introduces a specialized loss function that aligns text representations with this tree. To facilitate this learning, we developed RealBench, a comprehensive benchmark dataset that automatically incorporates a wide spectrum of hybrid texts produced through various human-AI collaboration processes. Our method improves performance in hybrid text detection tasks and significantly enhances robustness and generalization in out-of-distribution scenarios, particularly in few-shot learning conditions, further demonstrating the promise of training-based approaches in OOD settings. Our code and dataset are available at https://github.com/heyongxin233/DETree.
Paper Structure (63 sections, 1 theorem, 18 equations, 15 figures, 8 tables, 1 algorithm)

This paper contains 63 sections, 1 theorem, 18 equations, 15 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Detection of Purely-Generated Text.
  4. Detection of Hybrid or Collaborative Text.
  5. Method
  6. Motivation and RealBench Construction
  7. Hierarchical Affinity Tree Construction
  8. Tree-Structured Contrastive Learning
  9. Inference with DETree
  10. K-Means Based Database Compression.
  11. Retrieval-Based Few-Shot Adaptation.
  12. Experiments
  13. Experimental Setup
  14. Dataset.
  15. Comparison Methods and Metrics.
  16. ...and 48 more sections

Key Result

Theorem 3.1

[Hierarchical Similarity Constraint] If in the tree $\mathcal{T}$, for any leaf class X corresponding to node c, the following holds: Then, for any leaf classes X, Y, Z, the following inequalities holds: Where $\mathbb{E}[\mathrm{sim}(\cdot, \cdot)]$ denotes the expected similarity between categories.

Figures (15)

  • Figure 1: The overall framework of DETree. Class similarity matrix is computed from representations learned via supervised contrastive learning, with each class treated independently. Based on the similarity matrix, Hierarchical Affinity Tree (HAT) is constructed. Guided by the HAT, Tree-Structured Contrastive Loss is introduced to retrain the text encoder, aligning the representation space with the hierarchical relations defined by the HAT.
  • Figure 2: UMAP-based unsupervised visualization of the HART dataset in the representation space of the DETree. Colors indicate different text source types, with type indices corresponding to the hierarchical categorization defined in HART. Abbreviations of category names are defined in Figure \ref{['fig:heatmap']}.
  • Figure 3: Pairwise binary classification performance heatmap on the HART dataset, evaluated by TPR@5%FPR. Rows and columns represent the two text types involved in each detection pair. In the legend, MT denotes machine-generated text humanized by tools.
  • Figure 4: Robustness evaluation of different detection methods on the attack-augmented MAGE test set, measured by TPR@5%FPR. The x-axis denotes perturbation types. “No Attack” indicates that the model was trained without adversarial samples, while others were trained with all attack types. DETree is evaluated using a compressed 10K-sample version of RealBench as the database.
  • Figure 5: Detection performance (F1) on the MAGE test set using MAGE ( 500K) and RealBench ( 12M) as the database under different compression sizes. "ALL" indicates no compression.
  • ...and 10 more figures

Theorems & Definitions (1)

  • Theorem 3.1