DeepPavlov at SemEval-2024 Task 8: Leveraging Transfer Learning for Detecting Boundaries of Machine-Generated Texts

TL;DR

The paper tackles the boundary-detection problem in hybrid human-AI writing (Subtask C) by introducing a data augmentation pipeline that expands training data for supervised fine-tuning of DeBERTaV3. By evaluating RoBERTa, Longformer, and DeBERTaV3 under both original and augmented data, the authors show that augmentation significantly improves performance, with DeBERTaV3-large achieving a new best MAE of 13.375. The work highlights the importance of data diversity and powerful pretrained models for cross-domain robustness, and discusses domain-specific challenges (notably Outfox) as well as anomalies in machine-generated text that can aid boundary detection. The approach is positioned for potential multilingual extension and integration into DeepPavlov for practical applications in hybrid writing detection.

Abstract

The Multigenerator, Multidomain, and Multilingual Black-Box Machine-Generated Text Detection shared task in the SemEval-2024 competition aims to tackle the problem of misusing collaborative human-AI writing. Although there are a lot of existing detectors of AI content, they are often designed to give a binary answer and thus may not be suitable for more nuanced problem of finding the boundaries between human-written and machine-generated texts, while hybrid human-AI writing becomes more and more popular. In this paper, we address the boundary detection problem. Particularly, we present a pipeline for augmenting data for supervised fine-tuning of DeBERTaV3. We receive new best MAE score, according to the leaderboard of the competition, with this pipeline.

Figures (2)

• Figure 1: Statistics of the texts in the datasets
• Figure 2: Preprocessing for Augmentation Pipeline