Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Interpretable classification of wiki-review streams

Silvia García Méndez, Fátima Leal, Benedita Malheiro, Juan Carlos Burguillo Rial

TL;DR

The paper tackles real-time quality assurance in crowd-sourced wiki platforms by predicting which reviews will be reverted and explaining the rationale to editors. It introduces a stream-based, interpretable pipeline that combines offline feature analysis and synthetic data balancing with online incremental profiling and classification, supported by explainable outputs. Key contributions include a three-stage offline preprocessing workflow (feature analysis, engineering, selection), a synthetic data generator to address class imbalance, and an online framework using self-explainable models to justify decisions via natural language and graph-based explanations. Evaluated on Wikivoyage data, the method achieves near 90% performance across accuracy, precision, recall, and F-measure, demonstrating practical potential for real-time vandalism mitigation and editor feedback.

Abstract

Wiki articles are created and maintained by a crowd of editors, producing a continuous stream of reviews. Reviews can take the form of additions, reverts, or both. This crowdsourcing model is exposed to manipulation since neither reviews nor editors are automatically screened and purged. To protect articles against vandalism or damage, the stream of reviews can be mined to classify reviews and profile editors in real-time. The goal of this work is to anticipate and explain which reviews to revert. This way, editors are informed why their edits will be reverted. The proposed method employs stream-based processing, updating the profiling and classification models on each incoming event. The profiling uses side and content-based features employing Natural Language Processing, and editor profiles are incrementally updated based on their reviews. Since the proposed method relies on self-explainable classification algorithms, it is possible to understand why a review has been classified as a revert or a non-revert. In addition, this work contributes an algorithm for generating synthetic data for class balancing, making the final classification fairer. The proposed online method was tested with a real data set from Wikivoyage, which was balanced through the aforementioned synthetic data generation. The results attained near-90 % values for all evaluation metrics (accuracy, precision, recall, and F-measure).

Interpretable classification of wiki-review streams

TL;DR

The paper tackles real-time quality assurance in crowd-sourced wiki platforms by predicting which reviews will be reverted and explaining the rationale to editors. It introduces a stream-based, interpretable pipeline that combines offline feature analysis and synthetic data balancing with online incremental profiling and classification, supported by explainable outputs. Key contributions include a three-stage offline preprocessing workflow (feature analysis, engineering, selection), a synthetic data generator to address class imbalance, and an online framework using self-explainable models to justify decisions via natural language and graph-based explanations. Evaluated on Wikivoyage data, the method achieves near 90% performance across accuracy, precision, recall, and F-measure, demonstrating practical potential for real-time vandalism mitigation and editor feedback.

Abstract

Wiki articles are created and maintained by a crowd of editors, producing a continuous stream of reviews. Reviews can take the form of additions, reverts, or both. This crowdsourcing model is exposed to manipulation since neither reviews nor editors are automatically screened and purged. To protect articles against vandalism or damage, the stream of reviews can be mined to classify reviews and profile editors in real-time. The goal of this work is to anticipate and explain which reviews to revert. This way, editors are informed why their edits will be reverted. The proposed method employs stream-based processing, updating the profiling and classification models on each incoming event. The profiling uses side and content-based features employing Natural Language Processing, and editor profiles are incrementally updated based on their reviews. Since the proposed method relies on self-explainable classification algorithms, it is possible to understand why a review has been classified as a revert or a non-revert. In addition, this work contributes an algorithm for generating synthetic data for class balancing, making the final classification fairer. The proposed online method was tested with a real data set from Wikivoyage, which was balanced through the aforementioned synthetic data generation. The results attained near-90 % values for all evaluation metrics (accuracy, precision, recall, and F-measure).
Paper Structure (34 sections, 1 equation, 4 figures, 6 tables)

This paper contains 34 sections, 1 equation, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Profiling
  4. Analysis of reviews
  5. Vandalism detection
  6. Revert detection
  7. Quality prediction
  8. Transparency
  9. Fairness
  10. Research contribution
  11. Proposed method
  12. Offline processing
  13. Data pre-processing
  14. Feature analysis
  15. Feature engineering
  16. ...and 19 more sections

Figures (4)

  • Figure 1: Fair and transparent classification of Wikivoyage reviews as reverts and non-reverts.
  • Figure 2: Confusion matrix of the online rc classifier with balanced data.
  • Figure 3: Enhanced graph-based explanations built from the rf classifier. The non-revert and revert classes correspond to 0 and 1, respectively.
  • Figure 4: Graph-based explanations built from the rf classifier. The non-revert and revert classes correspond to 0 and 1, respectively.