Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Supply vs. Demand in Community-Based Fact-Checking on Social Media

Moritz Pilarski, Nicolas Pröllochs

TL;DR

This paper investigates the alignment of supply (fact-check notes) and demand (user requests) in community-based fact-checking on X's Community Notes. It leverages a large, one-year dataset of 1.1 million notes and requests to compare which posts are requested versus verified and to estimate how request displays alter contributor behavior using a quasi-experimental Cox model. The findings show demand concentrates on highly visible posts from influential accounts, while supply spans languages, sentiments, and topics more broadly, revealing a gap between what users want checked and what gets checked. Importantly, displaying requests significantly increases note creation by Top Writers, with hazard ratios up to $HR=1.79$ for later post ages and a global average effect $AMHR=1.32$, suggesting design interventions can improve alignment. These results have governance implications and point to future work integrating demand signals into platform monitoring and resource allocation.

Abstract

Fact-checking ecosystems on social media depend on the interplay between what users want checked and what contributors are willing to supply. Prior research has largely examined these forces in isolation, yet it remains unclear to what extent supply meets demand. We address this gap with an empirical analysis of a unique dataset of 1.1 million fact-checks and fact-checking requests from X's Community Notes platform between June 2024 and May 2025. We find that requests disproportionately target highly visible posts - those with more views and engagement and authored by influential accounts - whereas fact-checks are distributed more broadly across languages, sentiments, and topics. Using a quasi-experimental survival analysis, we further estimate the effect of displaying requests on subsequent note creation. Results show that requests significantly accelerate contributions from Top Writers. Altogether, our findings highlight a gap between the content that attracts requests for fact-checking and the content that ultimately receives fact-checks, while showing that user requests can steer contributors toward greater alignment. These insights carry important implications for platform governance and future research on online misinformation.

Supply vs. Demand in Community-Based Fact-Checking on Social Media

TL;DR

This paper investigates the alignment of supply (fact-check notes) and demand (user requests) in community-based fact-checking on X's Community Notes. It leverages a large, one-year dataset of 1.1 million notes and requests to compare which posts are requested versus verified and to estimate how request displays alter contributor behavior using a quasi-experimental Cox model. The findings show demand concentrates on highly visible posts from influential accounts, while supply spans languages, sentiments, and topics more broadly, revealing a gap between what users want checked and what gets checked. Importantly, displaying requests significantly increases note creation by Top Writers, with hazard ratios up to for later post ages and a global average effect , suggesting design interventions can improve alignment. These results have governance implications and point to future work integrating demand signals into platform monitoring and resource allocation.

Abstract

Fact-checking ecosystems on social media depend on the interplay between what users want checked and what contributors are willing to supply. Prior research has largely examined these forces in isolation, yet it remains unclear to what extent supply meets demand. We address this gap with an empirical analysis of a unique dataset of 1.1 million fact-checks and fact-checking requests from X's Community Notes platform between June 2024 and May 2025. We find that requests disproportionately target highly visible posts - those with more views and engagement and authored by influential accounts - whereas fact-checks are distributed more broadly across languages, sentiments, and topics. Using a quasi-experimental survival analysis, we further estimate the effect of displaying requests on subsequent note creation. Results show that requests significantly accelerate contributions from Top Writers. Altogether, our findings highlight a gap between the content that attracts requests for fact-checking and the content that ultimately receives fact-checks, while showing that user requests can steer contributors toward greater alignment. These insights carry important implications for platform governance and future research on online misinformation.
Paper Structure (15 sections, 4 equations, 8 figures)

This paper contains 15 sections, 4 equations, 8 figures.

Table of Contents

  1. Introduction
  2. Background
  3. Data & Methods
  4. Data Sources
  5. Data Collection
  6. Annotation of Language, Sentiment & Topic
  7. Empirical Analysis
  8. Supply vs. Demand (RQ 1)
  9. Effect of Request Displays on Note Creation (RQ 2)
  10. Discussion
  11. Conclusion
  12. Ethics Statement
  13. Request Display and Top Writer Status
  14. LLM-Based Annotation
  15. System Prompt

Figures (8)

  • Figure 1: Screenshots of (a) a post with a displayed Community Note and of (b) the Community Notes request dialog.
  • Figure 2: Split violin plots for the engagement metrics of posts targeted by note writers and requesters. Shown are kernel density estimates (areas), mean values (circles), and quartile borders (lines).
  • Figure 3: Split violin plots for the engagement metrics and account ages of users targeted by note writers and requesters. Shown are kernel density estimates (areas), mean values (circles), and quartile borders (lines)
  • Figure 5: Distribution of the media attachment status of posts targeted by note writers and requesters.
  • Figure 7: Distribution of the topics covered in posts targeted by note writers and requesters. Because topics are multi-/label, aggregated proportions do not sum to 100%.
  • ...and 3 more figures