Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LuxVeri at GenAI Detection Task 1: Inverse Perplexity Weighted Ensemble for Robust Detection of AI-Generated Text across English and Multilingual Contexts

Md Kamrujjaman Mobin, Md Saiful Islam

TL;DR

This paper tackles binary detection of AI-generated versus human-written text in both English and multilingual contexts. It introduces an inverse perplexity-weighted ensemble that combines three English models (RoBERTa-base, RoBERTa-base OpenAI detector, BERT-base-cased) and three multilingual models (RemBERT, XLM-RoBERTa-base, BERT-base-multilingual-cased) to boost robustness across languages. The method achieves Macro F1 scores of 0.7458 for English and 0.7513 for multilingual tasks, ranking 12th and 4th respectively, outperforming baselines and individual models. Overall, the study demonstrates that perplexity-aware ensembling can enhance detection performance in multilingual settings, while highlighting ongoing challenges with data imbalance and cross-language generalization and suggesting avenues for future improvement.

Abstract

This paper presents a system developed for Task 1 of the COLING 2025 Workshop on Detecting AI-Generated Content, focusing on the binary classification of machine-generated versus human-written text. Our approach utilizes an ensemble of models, with weights assigned according to each model's inverse perplexity, to enhance classification accuracy. For the English text detection task, we combined RoBERTa-base, RoBERTa-base with the OpenAI detector, and BERT-base-cased, achieving a Macro F1-score of 0.7458, which ranked us 12th out of 35 teams. We ensembled RemBERT, XLM-RoBERTa-base, and BERT-base-multilingual-case for the multilingual text detection task, employing the same inverse perplexity weighting technique. This resulted in a Macro F1-score of 0.7513, positioning us 4th out of 25 teams. Our results demonstrate the effectiveness of inverse perplexity weighting in improving the robustness of machine-generated text detection across both monolingual and multilingual settings, highlighting the potential of ensemble methods for this challenging task.

LuxVeri at GenAI Detection Task 1: Inverse Perplexity Weighted Ensemble for Robust Detection of AI-Generated Text across English and Multilingual Contexts

TL;DR

This paper tackles binary detection of AI-generated versus human-written text in both English and multilingual contexts. It introduces an inverse perplexity-weighted ensemble that combines three English models (RoBERTa-base, RoBERTa-base OpenAI detector, BERT-base-cased) and three multilingual models (RemBERT, XLM-RoBERTa-base, BERT-base-multilingual-cased) to boost robustness across languages. The method achieves Macro F1 scores of 0.7458 for English and 0.7513 for multilingual tasks, ranking 12th and 4th respectively, outperforming baselines and individual models. Overall, the study demonstrates that perplexity-aware ensembling can enhance detection performance in multilingual settings, while highlighting ongoing challenges with data imbalance and cross-language generalization and suggesting avenues for future improvement.

Abstract

This paper presents a system developed for Task 1 of the COLING 2025 Workshop on Detecting AI-Generated Content, focusing on the binary classification of machine-generated versus human-written text. Our approach utilizes an ensemble of models, with weights assigned according to each model's inverse perplexity, to enhance classification accuracy. For the English text detection task, we combined RoBERTa-base, RoBERTa-base with the OpenAI detector, and BERT-base-cased, achieving a Macro F1-score of 0.7458, which ranked us 12th out of 35 teams. We ensembled RemBERT, XLM-RoBERTa-base, and BERT-base-multilingual-case for the multilingual text detection task, employing the same inverse perplexity weighting technique. This resulted in a Macro F1-score of 0.7513, positioning us 4th out of 25 teams. Our results demonstrate the effectiveness of inverse perplexity weighting in improving the robustness of machine-generated text detection across both monolingual and multilingual settings, highlighting the potential of ensemble methods for this challenging task.
Paper Structure (15 sections, 3 equations, 1 figure, 6 tables)

This paper contains 15 sections, 3 equations, 1 figure, 6 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Dataset
  4. Related Work
  5. System Overview
  6. Ensemble Model Selection and Justification
  7. Data Pre-processing
  8. Training Procedure
  9. Ensemble Voting Strategy
  10. Perplexity Calculation
  11. Perplexity-Based Weighting Adjustment
  12. Weighted Soft-Voting
  13. Results
  14. Discussion and Conclusion
  15. Appendix

Figures (1)

  • Figure 1: Overall Framework of our Proposed System.