Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Wikipedia is Not a Dictionary, Delete! Text Classification as a Proxy for Analysing Wiki Deletion Discussions

Hsuvas Borkakoty, Luis Espinosa-Anke

TL;DR

This work tackles automated content moderation in wiki ecosystems by constructing a multilingual, multi-platform deletion-discussion corpus and evaluating a range of language models on outcome, stance, and policy prediction tasks. It demonstrates that deletion discussions are comparatively easier to predict than other outcomes and shows that self-reported tags have inconsistent utility, especially when masked. The paper reports strong baselines from RoBERTa and XLM-R models, with SetFit offering notable gains on small datasets and in cross-platform settings, highlighting the value of efficient, cross-domain approaches. The findings have practical implications for deploying automated moderation tools across diverse wiki communities and languages, informing model choice and deployment strategies in production systems.

Abstract

Automated content moderation for collaborative knowledge hubs like Wikipedia or Wikidata is an important yet challenging task due to multiple factors. In this paper, we construct a database of discussions happening around articles marked for deletion in several Wikis and in three languages, which we then use to evaluate a range of LMs on different tasks (from predicting the outcome of the discussion to identifying the implicit policy an individual comment might be pointing to). Our results reveal, among others, that discussions leading to deletion are easier to predict, and that, surprisingly, self-produced tags (keep, delete or redirect) don't always help guiding the classifiers, presumably because of users' hesitation or deliberation within comments.

Wikipedia is Not a Dictionary, Delete! Text Classification as a Proxy for Analysing Wiki Deletion Discussions

TL;DR

This work tackles automated content moderation in wiki ecosystems by constructing a multilingual, multi-platform deletion-discussion corpus and evaluating a range of language models on outcome, stance, and policy prediction tasks. It demonstrates that deletion discussions are comparatively easier to predict than other outcomes and shows that self-reported tags have inconsistent utility, especially when masked. The paper reports strong baselines from RoBERTa and XLM-R models, with SetFit offering notable gains on small datasets and in cross-platform settings, highlighting the value of efficient, cross-domain approaches. The findings have practical implications for deploying automated moderation tools across diverse wiki communities and languages, informing model choice and deployment strategies in production systems.

Abstract

Automated content moderation for collaborative knowledge hubs like Wikipedia or Wikidata is an important yet challenging task due to multiple factors. In this paper, we construct a database of discussions happening around articles marked for deletion in several Wikis and in three languages, which we then use to evaluate a range of LMs on different tasks (from predicting the outcome of the discussion to identifying the implicit policy an individual comment might be pointing to). Our results reveal, among others, that discussions leading to deletion are easier to predict, and that, surprisingly, self-produced tags (keep, delete or redirect) don't always help guiding the classifiers, presumably because of users' hesitation or deliberation within comments.

Paper Structure

This paper contains 15 sections, 5 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Tasks and dataset Construction
  3. Outcome Prediction
  4. Dataset for Stance Detection and Policy Prediction
  5. Experiments
  6. Outcome Prediction
  7. In-platform
  8. Cross-platform
  9. Multilingual
  10. Stance and Policy prediction
  11. Stance Detection
  12. Policy Prediction
  13. Conclusion
  14. Train/Validation/Test Splits
  15. Training details and results for Outcome Prediction

Figures (5)

  • Figure 1: Example of a Deletion Discussion in English Wikipedia
  • Figure 2: Confusion Matrix for RoBERTa-Large model in Outcome Prediction Task.
  • Figure 3: Normalized perplexity distribution for Twitter-XLM-Roberta in a sample of the Greek Wikipedia.
  • Figure 4: Confusion Matrix for XLM-RoBERTa-Large model in Outcome Prediction Task for Spanish and Greek Wikipedia.
  • Figure 5: Confusion Matrix for RoBERTa-Large in Policy Prediction.