LuxVeri at GenAI Detection Task 3: Cross-Domain Detection of AI-Generated Text Using Inverse Perplexity-Weighted Ensemble of Fine-Tuned Transformer Models
Md Kamrujjaman Mobin, Md Saiful Islam
TL;DR
This work tackles cross-domain generation-detection (GenAI Task 3) by proposing an ensemble of fine-tuned RoBERTa-base detectors, including an OpenAI detector-integrated variant, guided by inverse perplexity weighting to boost high-confidence predictions. Using the RAID dataset with a balanced ~10% training subset, the approach achieves AGG TPRs of 0.826 in non-adversarial and 0.801 in adversarial cross-domain MGT detection, ranking 10th and 8th respectively. The results demonstrate that inverse perplexity weighting improves generalization across diverse domains and attack scenarios, affirming the viability of transformer-based detectors for cross-domain AI-generated content detection. The work also discusses limitations and avenues for extending multilingual coverage and developing more efficient ensemble strategies.
Abstract
This paper presents our approach for Task 3 of the GenAI content detection workshop at COLING-2025, focusing on Cross-Domain Machine-Generated Text (MGT) Detection. We propose an ensemble of fine-tuned transformer models, enhanced by inverse perplexity weighting, to improve classification accuracy across diverse text domains. For Subtask A (Non-Adversarial MGT Detection), we combined a fine-tuned RoBERTa-base model with an OpenAI detector-integrated RoBERTa-base model, achieving an aggregate TPR score of 0.826, ranking 10th out of 23 detectors. In Subtask B (Adversarial MGT Detection), our fine-tuned RoBERTa-base model achieved a TPR score of 0.801, securing 8th out of 22 detectors. Our results demonstrate the effectiveness of inverse perplexity-based weighting for enhancing generalization and performance in both non-adversarial and adversarial MGT detection, highlighting the potential for transformer models in cross-domain AI-generated content detection.