Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DuoLens: A Framework for Robust Detection of Machine-Generated Multilingual Text and Code

Shriyansh Agrawal, Aidan Lau, Sanyam Shah, Ahan M R, Kevin Zhu, Sunishchal Dev, Vasu Sharma

TL;DR

DuoLens tackles the challenge of detecting machine-generated multilingual text and code by leveraging encoder-only small language models fine-tuned on carefully constructed, balanced multilingual and code datasets. It introduces a dual-encoder architecture that fuses CodeBERT and CodeBERTa, coupled with fine-tuned multilingual encoders, to achieve high detection performance while significantly reducing compute and memory requirements compared to LLM-based detectors. The approach yields AUROC $=0.97$ to $0.99$ and macro-F1 $=0.89$ to $0.94$, with latency improvements of $8$-$12\times$ and peak VRAM reductions of $3$-$5\times$ at $512$-token inputs, and preserves robustness under paraphrase/back-translation transformations. The work provides large, reproducible datasets and a transparent evaluation protocol, demonstrating strong cross-language and cross-model generalization and offering a practical, efficient solution for detectors across text and code domains.

Abstract

The prevalence of Large Language Models (LLMs) for generating multilingual text and source code has only increased the imperative for machine-generated content detectors to be accurate and efficient across domains. Current detectors, predominantly utilizing zero-shot methods, such as Fast DetectGPT or GPTZero, either incur high computational cost or lack sufficient accuracy, often with a trade-off between the two, leaving room for further improvement. To address these gaps, we propose the fine-tuning of encoder-only Small Language Models (SLMs), in particular, the pre-trained models of RoBERTA and CodeBERTa using specialized datasets on source code and other natural language to prove that for the task of binary classification, SLMs outperform LLMs by a huge margin whilst using a fraction of compute. Our encoders achieve AUROC $= 0.97$ to $0.99$ and macro-F1 $0.89$ to $0.94$ while reducing latency by $8$-$12\times$ and peak VRAM by $3$-$5\times$ at $512$-token inputs. Under cross-generator shifts and adversarial transformations (paraphrase, back-translation; code formatting/renaming), performance retains $\geq 92%$ of clean AUROC. We release training and evaluation scripts with seeds and configs; a reproducibility checklist is also included.

DuoLens: A Framework for Robust Detection of Machine-Generated Multilingual Text and Code

TL;DR

DuoLens tackles the challenge of detecting machine-generated multilingual text and code by leveraging encoder-only small language models fine-tuned on carefully constructed, balanced multilingual and code datasets. It introduces a dual-encoder architecture that fuses CodeBERT and CodeBERTa, coupled with fine-tuned multilingual encoders, to achieve high detection performance while significantly reducing compute and memory requirements compared to LLM-based detectors. The approach yields AUROC to and macro-F1 to , with latency improvements of - and peak VRAM reductions of - at -token inputs, and preserves robustness under paraphrase/back-translation transformations. The work provides large, reproducible datasets and a transparent evaluation protocol, demonstrating strong cross-language and cross-model generalization and offering a practical, efficient solution for detectors across text and code domains.

Abstract

The prevalence of Large Language Models (LLMs) for generating multilingual text and source code has only increased the imperative for machine-generated content detectors to be accurate and efficient across domains. Current detectors, predominantly utilizing zero-shot methods, such as Fast DetectGPT or GPTZero, either incur high computational cost or lack sufficient accuracy, often with a trade-off between the two, leaving room for further improvement. To address these gaps, we propose the fine-tuning of encoder-only Small Language Models (SLMs), in particular, the pre-trained models of RoBERTA and CodeBERTa using specialized datasets on source code and other natural language to prove that for the task of binary classification, SLMs outperform LLMs by a huge margin whilst using a fraction of compute. Our encoders achieve AUROC to and macro-F1 to while reducing latency by - and peak VRAM by - at -token inputs. Under cross-generator shifts and adversarial transformations (paraphrase, back-translation; code formatting/renaming), performance retains of clean AUROC. We release training and evaluation scripts with seeds and configs; a reproducibility checklist is also included.
Paper Structure (44 sections, 4 figures, 3 tables)

This paper contains 44 sections, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Contributions:
  3. Related Work
  4. Pre-existing Datasets.
  5. RoBERTa as a detector.
  6. Methodology
  7. Multilingual Dataset.
  8. Code Dataset.
  9. Dataset Creation
  10. Architecture.
  11. Multilingual encoders.
  12. Experiments
  13. Experimental Setup
  14. Baselines.
  15. Metrics.
  16. ...and 29 more sections

Figures (4)

  • Figure 1: Overview of the experimental setup where each sample from the dataset is processed by the models and classified as either 0 (human-written) or 1 (machine-generated)
  • Figure 2: An overview of the methodology adopted in this work is as follows. We first constructed new datasets by curating and extending samples from existing resources. These datasets were subsequently employed to fine-tune and evaluate the models that constitute DuoLens.
  • Figure 3: Source code dataset visualizations
  • Figure 4: Multilingual text dataset visualizations