Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Authorship Obfuscation in Multilingual Machine-Generated Text Detection

Dominik Macko, Robert Moro, Adaku Uchendu, Ivan Srba, Jason Samuel Lucas, Michiharu Yamashita, Nafis Irtiza Tripto, Dongwon Lee, Jakub Simko, Maria Bielikova

TL;DR

The paper addresses the vulnerability of multilingual machine-generated text detectors to authorship obfuscation (AO). It introduces the first comprehensive multilingual benchmark of AO methods across 11 languages, attacking 37 detectors with 10 automated AO techniques, and provides a dataset of about 740k obfuscated samples. Key findings show that homoglyph attacks are among the most effective across languages, though some AO methods drastically degrade readability; English-only detectors are particularly susceptible, while multilingual detectors show improved robustness. The study further demonstrates that simple data augmentation with obfuscated data can meaningfully boost adversarial robustness, and it releases a large-scale obfuscated-text resource to support future research. Overall, the work highlights cross-language vulnerabilities and proposes practical defense avenues through preprocessing and adversarial retraining, informing safer deployment of multilingual MGT-detection systems.

Abstract

High-quality text generation capability of recent Large Language Models (LLMs) causes concerns about their misuse (e.g., in massive generation/spread of disinformation). Machine-generated text (MGT) detection is important to cope with such threats. However, it is susceptible to authorship obfuscation (AO) methods, such as paraphrasing, which can cause MGTs to evade detection. So far, this was evaluated only in monolingual settings. Thus, the susceptibility of recently proposed multilingual detectors is still unknown. We fill this gap by comprehensively benchmarking the performance of 10 well-known AO methods, attacking 37 MGT detection methods against MGTs in 11 languages (i.e., 10 $\times$ 37 $\times$ 11 = 4,070 combinations). We also evaluate the effect of data augmentation on adversarial robustness using obfuscated texts. The results indicate that all tested AO methods can cause evasion of automated detection in all tested languages, where homoglyph attacks are especially successful. However, some of the AO methods severely damaged the text, making it no longer readable or easily recognizable by humans (e.g., changed language, weird characters).

Authorship Obfuscation in Multilingual Machine-Generated Text Detection

TL;DR

The paper addresses the vulnerability of multilingual machine-generated text detectors to authorship obfuscation (AO). It introduces the first comprehensive multilingual benchmark of AO methods across 11 languages, attacking 37 detectors with 10 automated AO techniques, and provides a dataset of about 740k obfuscated samples. Key findings show that homoglyph attacks are among the most effective across languages, though some AO methods drastically degrade readability; English-only detectors are particularly susceptible, while multilingual detectors show improved robustness. The study further demonstrates that simple data augmentation with obfuscated data can meaningfully boost adversarial robustness, and it releases a large-scale obfuscated-text resource to support future research. Overall, the work highlights cross-language vulnerabilities and proposes practical defense avenues through preprocessing and adversarial retraining, informing safer deployment of multilingual MGT-detection systems.

Abstract

High-quality text generation capability of recent Large Language Models (LLMs) causes concerns about their misuse (e.g., in massive generation/spread of disinformation). Machine-generated text (MGT) detection is important to cope with such threats. However, it is susceptible to authorship obfuscation (AO) methods, such as paraphrasing, which can cause MGTs to evade detection. So far, this was evaluated only in monolingual settings. Thus, the susceptibility of recently proposed multilingual detectors is still unknown. We fill this gap by comprehensively benchmarking the performance of 10 well-known AO methods, attacking 37 MGT detection methods against MGTs in 11 languages (i.e., 10 37 11 = 4,070 combinations). We also evaluate the effect of data augmentation on adversarial robustness using obfuscated texts. The results indicate that all tested AO methods can cause evasion of automated detection in all tested languages, where homoglyph attacks are especially successful. However, some of the AO methods severely damaged the text, making it no longer readable or easily recognizable by humans (e.g., changed language, weird characters).
Paper Structure (25 sections, 3 figures, 21 tables)

This paper contains 25 sections, 3 figures, 21 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. MGT Detection
  4. MGT Obfuscation
  5. Methodology
  6. Authorship Obfuscation Methods
  7. Dataset
  8. MGT Detection Methods
  9. Experiments and Results
  10. Multilingual Capability of AO Methods
  11. Human Validation of Readability
  12. Adversarial Robustness of Multilingual MGT Detection Methods
  13. Multilingual Obfuscation for Data Augmentation
  14. Discussion
  15. Conclusion
  16. ...and 10 more sections

Figures (3)

  • Figure 1: Benchmarking authorship obfuscation techniques for machine-generated text detection.
  • Figure 2: Detection performance (AUC ROC) of originally (leftmost bar) and adversarially trained MGT detection methods on the test data including all unobfuscated texts and obfuscated texts by AO methods passing the quality check. Note that the y-axis starts at 0.8.
  • Figure 3: Ablation of detection performance (AUC ROC) of originally (leftmost bar) and adversarially trained MGT detection methods on the high-quality test data. Note that the y-axis starts at 0.8.