Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multilingual Reference Need Assessment System for Wikipedia

Aitolkyn Baigutanova, Francisco Navas, Pablo Aragon, Mykola Trokhymovych, Muniza Aslam, Ai-Jou Chou, Miriam Redi, Diego Saez-Trumper

Abstract

Wikipedia is a critical source of information for millions of users across the Web. It serves as a key resource for large language models, search engines, question-answering systems, and other Web-based applications. In Wikipedia, content needs to be verifiable, meaning that readers can check that claims are backed by references to reliable sources. This depends on manual verification by editors, an effective but labor-intensive process, especially given the high volume of daily edits. To address this challenge, we introduce a multilingual machine learning system to assist editors in identifying claims requiring citations. Our approach is tested in 10 language editions of Wikipedia, outperforming existing benchmarks for reference need assessment. We not only consider machine learning evaluation metrics but also system requirements, allowing us to explore the trade-offs between model accuracy and computational efficiency under real-world infrastructure constraints. We deploy our system in production and release data and code to support further research.

Multilingual Reference Need Assessment System for Wikipedia

Abstract

Wikipedia is a critical source of information for millions of users across the Web. It serves as a key resource for large language models, search engines, question-answering systems, and other Web-based applications. In Wikipedia, content needs to be verifiable, meaning that readers can check that claims are backed by references to reliable sources. This depends on manual verification by editors, an effective but labor-intensive process, especially given the high volume of daily edits. To address this challenge, we introduce a multilingual machine learning system to assist editors in identifying claims requiring citations. Our approach is tested in 10 language editions of Wikipedia, outperforming existing benchmarks for reference need assessment. We not only consider machine learning evaluation metrics but also system requirements, allowing us to explore the trade-offs between model accuracy and computational efficiency under real-world infrastructure constraints. We deploy our system in production and release data and code to support further research.
Paper Structure (24 sections, 1 equation, 8 figures, 2 tables)

This paper contains 24 sections, 1 equation, 8 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. References on Wikipedia
  4. Masked Language Models
  5. Large Language Models
  6. Reference Need
  7. System Design and Deployment
  8. System Requirements
  9. Data Collection
  10. Data Processing
  11. Data Splitting
  12. Model Training
  13. Model Deployment
  14. Experiments
  15. Input Selection
  16. ...and 9 more sections

Figures (8)

  • Figure 1: The Reference Need pipeline to compute the reference coverage score of an article revision.
  • Figure 2: Proportion of sentences accompanied by a citation in featured articles by language.
  • Figure 3: Data sample of a sentence that needs a citation.
  • Figure 4: Performance and latency comparison. The red dot in all subfigures represents the fine-tuned distil-128 model.
  • Figure 5: Performance by language using distilbert-128.
  • ...and 3 more figures