Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

WiDe-analysis: Enabling One-click Content Moderation Analysis on Wikipedia's Articles for Deletion

Hsuvas Borkakoty, Luis Espinosa-Anke

TL;DR

WiDe-analysis introduces a unified Python toolkit for one-click analysis of Wikipedia AfD discussions, combining data gathering, preprocessing, and NLP analyses for outcome, stance, and policy prediction, as well as sentiment and offensive-language detection. The work presents a substantial experimental evaluation showing RoBERTa-Large as the most robust performer across tasks, with LLMs underperforming relative to fine-tuned transformers in outcome prediction. A HuggingFace Space provides an accessible demo, while WiDe-powered insights demonstrate interpretable connections between textual signals and moderation outcomes. By releasing data, models, and tooling, the work aims to accelerate research on content moderation in Wikipedia and beyond, with broad applicability to runtime analysis of online discussions and policy-grounded explanations.

Abstract

Content moderation in online platforms is crucial for ensuring activity therein adheres to existing policies, especially as these platforms grow. NLP research in this area has typically focused on automating some part of it given that it is not feasible to monitor all active discussions effectively. Past works have focused on revealing deletion patterns with like sentiment analysis, or on developing platform-specific models such as Wikipedia policy or stance detectors. Unsurprisingly, however, this valuable body of work is rather scattered, with little to no agreement with regards to e.g., the deletion discussions corpora used for training or the number of stance labels. Moreover, while efforts have been made to connect stance with rationales (e.g., to ground a deletion decision on the relevant policy), there is little explanability work beyond that. In this paper, we introduce a suite of experiments on Wikipedia deletion discussions and wide-analyis (Wikipedia Deletion Analysis), a Python package aimed at providing one click analysis to content moderation discussions. We release all assets associated with wide-analysis, including data, models and the Python package, and a HuggingFace space with the goal to accelerate research on automating content moderation in Wikipedia and beyond.

WiDe-analysis: Enabling One-click Content Moderation Analysis on Wikipedia's Articles for Deletion

TL;DR

WiDe-analysis introduces a unified Python toolkit for one-click analysis of Wikipedia AfD discussions, combining data gathering, preprocessing, and NLP analyses for outcome, stance, and policy prediction, as well as sentiment and offensive-language detection. The work presents a substantial experimental evaluation showing RoBERTa-Large as the most robust performer across tasks, with LLMs underperforming relative to fine-tuned transformers in outcome prediction. A HuggingFace Space provides an accessible demo, while WiDe-powered insights demonstrate interpretable connections between textual signals and moderation outcomes. By releasing data, models, and tooling, the work aims to accelerate research on content moderation in Wikipedia and beyond, with broad applicability to runtime analysis of online discussions and policy-grounded explanations.

Abstract

Content moderation in online platforms is crucial for ensuring activity therein adheres to existing policies, especially as these platforms grow. NLP research in this area has typically focused on automating some part of it given that it is not feasible to monitor all active discussions effectively. Past works have focused on revealing deletion patterns with like sentiment analysis, or on developing platform-specific models such as Wikipedia policy or stance detectors. Unsurprisingly, however, this valuable body of work is rather scattered, with little to no agreement with regards to e.g., the deletion discussions corpora used for training or the number of stance labels. Moreover, while efforts have been made to connect stance with rationales (e.g., to ground a deletion decision on the relevant policy), there is little explanability work beyond that. In this paper, we introduce a suite of experiments on Wikipedia deletion discussions and wide-analyis (Wikipedia Deletion Analysis), a Python package aimed at providing one click analysis to content moderation discussions. We release all assets associated with wide-analysis, including data, models and the Python package, and a HuggingFace space with the goal to accelerate research on automating content moderation in Wikipedia and beyond.
Paper Structure (15 sections, 6 figures, 5 tables)

This paper contains 15 sections, 6 figures, 5 tables.

Table of Contents

  1. Introduction
  2. NLP on Articles for Deletion
  3. Outcome prediction
  4. Stance and policy prediction
  5. The WiDe-analysis Python package
  6. Gathering and analyzing the data
  7. Text analysis models
  8. WiDe-Analysis Hugging Face Space
  9. WiDe-powered insights into deletion discussions
  10. Conclusion
  11. Limitations
  12. Ethics statement
  13. Confounding Factors in Outcome Prediction
  14. WiDe-Analysis Huggingface space
  15. Prompts for Outcome prediction

Figures (6)

  • Figure 1: A Wikipedia discussion on an Article for Deletion (AfD) and an examples showing how the wide-analysis package enables one-click data retrieval and analysis.
  • Figure 2: Category-wise discussion length and Sentence length Distribution for Outcome Prediction Dataset
  • Figure 3: Correlation between sentiments and the labels
  • Figure 4: Correlation between stance and the labels
  • Figure 5: Confusion Matrix for RoBERTa-Large model in Outcome Prediction Task
  • ...and 1 more figures