Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Comprehensive Framework for Semantic Similarity Analysis of Human and AI-Generated Text Using Transformer Architectures and Ensemble Techniques

Lifu Gao, Ziwei Liu, Qi Zhang

TL;DR

The paper tackles AI-generated text detection by reframing the problem as semantic similarity analysis, arguing that semantic structure differentiates human and machine writing more than surface features. It introduces a multi-layer architecture built on DeBERTa-v3-large, enhanced with Bi-LSTM, linear attention pooling, and adversarial robustness tricks, plus sector-context augmentation and dynamic target shuffling to improve generalization. An extensive ensemble strategy, including Electra-based variants and wide-output configurations, combined with a Pearson-based loss and auxiliary MSE, yields state-of-the-art performance across Pearson correlation, MSE, F1, and AUC on diverse data. The work demonstrates strong practical potential for AI-generated text detection and related semantic-difference tasks, with future directions toward domain-specific pretraining and broader augmentation techniques for even better generalization.

Abstract

The rapid advancement of large language models (LLMs) has made detecting AI-generated text an increasingly critical challenge. Traditional methods often fail to capture the nuanced semantic differences between human and machine-generated content. We therefore propose a novel approach based on semantic similarity analysis, leveraging a multi-layered architecture that combines a pre-trained DeBERTa-v3-large model, Bi-directional LSTMs, and linear attention pooling to capture both local and global semantic patterns. To enhance performance, we employ advanced input and output augmentation techniques such as sector-level context integration and wide output configurations. These techniques enable the model to learn more discriminative features and generalize across diverse domains. Experimental results show that this approach works better than traditional methods, proving its usefulness for AI-generated text detection and other text comparison tasks.

A Comprehensive Framework for Semantic Similarity Analysis of Human and AI-Generated Text Using Transformer Architectures and Ensemble Techniques

TL;DR

The paper tackles AI-generated text detection by reframing the problem as semantic similarity analysis, arguing that semantic structure differentiates human and machine writing more than surface features. It introduces a multi-layer architecture built on DeBERTa-v3-large, enhanced with Bi-LSTM, linear attention pooling, and adversarial robustness tricks, plus sector-context augmentation and dynamic target shuffling to improve generalization. An extensive ensemble strategy, including Electra-based variants and wide-output configurations, combined with a Pearson-based loss and auxiliary MSE, yields state-of-the-art performance across Pearson correlation, MSE, F1, and AUC on diverse data. The work demonstrates strong practical potential for AI-generated text detection and related semantic-difference tasks, with future directions toward domain-specific pretraining and broader augmentation techniques for even better generalization.

Abstract

The rapid advancement of large language models (LLMs) has made detecting AI-generated text an increasingly critical challenge. Traditional methods often fail to capture the nuanced semantic differences between human and machine-generated content. We therefore propose a novel approach based on semantic similarity analysis, leveraging a multi-layered architecture that combines a pre-trained DeBERTa-v3-large model, Bi-directional LSTMs, and linear attention pooling to capture both local and global semantic patterns. To enhance performance, we employ advanced input and output augmentation techniques such as sector-level context integration and wide output configurations. These techniques enable the model to learn more discriminative features and generalize across diverse domains. Experimental results show that this approach works better than traditional methods, proving its usefulness for AI-generated text detection and other text comparison tasks.
Paper Structure (24 sections, 18 equations, 5 figures, 1 table)

This paper contains 24 sections, 18 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Transformer Backbone
  5. Bidirectional LSTM Enhancement
  6. Linear Attention Pooling
  7. Fully Connected Layer
  8. Alternative Model Architectures
  9. Electra-Based Models
  10. Wide Output Configurations
  11. Bi-LSTM and Sector Contexts
  12. Loss Function
  13. Data Preprocessing
  14. Target Grouping and Stratification
  15. Dynamic Target Shuffling
  16. ...and 9 more sections

Figures (5)

  • Figure 1: The pipline of transformer-based architectures model.
  • Figure 2: The pipline of Bidirectional LSTM Enhancement.
  • Figure 3: The linear attention pooling.
  • Figure 4: Semantic similarity average score graph.
  • Figure 5: Performance indicator change chart