Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FactCheck Editor: Multilingual Text Editor with End-to-End fact-checking

Vinay Setty

TL;DR

This work presents FactCheck Editor, a web-based, multilingual text editor that enables end-to-end fact-checking by detecting check-worthy claims, retrieving diverse web-based evidence, and predicting veracity via NLI, complemented by LLM-generated justifications and claim-revision suggestions. The system leverages a multilingual pipeline with fine-tuned transformers (e.g., XLM-RoBERTa-Large) for claim detection and veracity prediction, and employs Mistral-7b and other LLMs for query generation and justification. Experimental results show the multilingual fine-tuned models outperform large LLM baselines on claim detection and veracity prediction across 90+ languages, while LLMs excel at generation tasks such as summarization and correction suggestions. The work demonstrates a practical, end-to-end multilingual fact-checking workflow with a live demonstration and highlights the potential impact for editors in newsrooms and public discourse, while outlining future directions for broader multilingual coverage and evaluation.

Abstract

We introduce 'FactCheck Editor', an advanced text editor designed to automate fact-checking and correct factual inaccuracies. Given the widespread issue of misinformation, often a result of unintentional mistakes by content creators, our tool aims to address this challenge. It supports over 90 languages and utilizes transformer models to assist humans in the labor-intensive process of fact verification. This demonstration showcases a complete workflow that detects text claims in need of verification, generates relevant search engine queries, and retrieves appropriate documents from the web. It employs Natural Language Inference (NLI) to predict the veracity of claims and uses LLMs to summarize the evidence and suggest textual revisions to correct any errors in the text. Additionally, the effectiveness of models used in claim detection and veracity assessment is evaluated across multiple languages.

FactCheck Editor: Multilingual Text Editor with End-to-End fact-checking

TL;DR

This work presents FactCheck Editor, a web-based, multilingual text editor that enables end-to-end fact-checking by detecting check-worthy claims, retrieving diverse web-based evidence, and predicting veracity via NLI, complemented by LLM-generated justifications and claim-revision suggestions. The system leverages a multilingual pipeline with fine-tuned transformers (e.g., XLM-RoBERTa-Large) for claim detection and veracity prediction, and employs Mistral-7b and other LLMs for query generation and justification. Experimental results show the multilingual fine-tuned models outperform large LLM baselines on claim detection and veracity prediction across 90+ languages, while LLMs excel at generation tasks such as summarization and correction suggestions. The work demonstrates a practical, end-to-end multilingual fact-checking workflow with a live demonstration and highlights the potential impact for editors in newsrooms and public discourse, while outlining future directions for broader multilingual coverage and evaluation.

Abstract

We introduce 'FactCheck Editor', an advanced text editor designed to automate fact-checking and correct factual inaccuracies. Given the widespread issue of misinformation, often a result of unintentional mistakes by content creators, our tool aims to address this challenge. It supports over 90 languages and utilizes transformer models to assist humans in the labor-intensive process of fact verification. This demonstration showcases a complete workflow that detects text claims in need of verification, generates relevant search engine queries, and retrieves appropriate documents from the web. It employs Natural Language Inference (NLI) to predict the veracity of claims and uses LLMs to summarize the evidence and suggest textual revisions to correct any errors in the text. Additionally, the effectiveness of models used in claim detection and veracity assessment is evaluated across multiple languages.
Paper Structure (23 sections, 4 figures, 2 tables)

This paper contains 23 sections, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. System Overview
  4. Check-worthy Claim Detection
  5. Sentence segmentation and Co-reference resolution:
  6. Claim classification:
  7. Evidence Retrieval
  8. Query generation:
  9. Retrieval from search engines:
  10. Deduplication and Snippet Extraction
  11. Veracity Prediction
  12. Natural Language Inference (NLI):
  13. Justification Generation and Claim Fix Suggestion:
  14. Experimental Evaluation
  15. Claim Detection
  16. ...and 8 more sections

Figures (4)

  • Figure 1: System Architecture of FactCheck Editor
  • Figure 2: Evaluation of claim detection for 118 langauges usig XLM-RoBERTa-Large, GPT-3.5-Turbo and Mistral-7b.
  • Figure 3: Evaluation of veracity prediction for 46 languages.
  • Figure 4: FactCheck Editor demonstration