Can Community Notes Replace Professional Fact-Checkers?

TL;DR

The paper investigates whether community notes can substitute for professional fact-checkers on Twitter/X by analyzing how often notes cite fact-checking sources and how note strategies vary with claim stakes. Using a large-scale annotation pipeline, including GPT-4-based topic classification and manual validation, the study finds that professional fact-checkers underpin a nontrivial portion of notes, especially for health and political topics and for claims connected to broader narratives. The results argue against the notion that community moderation alone suffices for high-quality misinformation debunking, showing a deep, complementary interdependence between citizen and professional fact-checking. The work highlights policy implications: defunding or eroding professional fact-checking could degrade note quality and undermine information integrity, while suggesting a cooperative model that leverages automated, human-in-the-loop fact-checking to scale across platforms.

Abstract

Two commonly employed strategies to combat the rise of misinformation on social media are (i) fact-checking by professional organisations and (ii) community moderation by platform users. Policy changes by Twitter/X and, more recently, Meta, signal a shift away from partnerships with fact-checking organisations and towards an increased reliance on crowdsourced community notes. However, the extent and nature of dependencies between fact-checking and helpful community notes remain unclear. To address these questions, we use language models to annotate a large corpus of Twitter/X community notes with attributes such as topic, cited sources, and whether they refute claims tied to broader misinformation narratives. Our analysis reveals that community notes cite fact-checking sources up to five times more than previously reported. Fact-checking is especially crucial for notes on posts linked to broader narratives, which are twice as likely to reference fact-checking sources compared to other sources. Our results show that successful community moderation relies on professional fact-checking and highlight how citizen and professional fact-checking are deeply intertwined.

Figures (8)

• Figure 1: An example of a community note. Notice the fact-checking link and rating.
• Figure 2: The categories of links used by Community notes' authors as a source. a) all community notes; b) Community notes rated as 'helpful'; c) community notes rated as 'unhelpful'. Notice the 'fact-checking' category.
• Figure 3: Mean scores of community annotations of misleading posts.
• Figure 4: (a) strategies in debunking claims related to broader narratives. (b) the different ways in which fact-checking sources are used to debunk claims.
• Figure 5: Distribution of notes' topics, with and without a fact-checking source.