Hyperactive Minority Alter the Stability of Community Notes

TL;DR

The paper addresses whether community-based notes on X decentralize epistemic authority or reproduce platform-driven dynamics by analyzing the full public CNs dataset (2021–2025) and the production consensus algorithm. It finds strong participation inequality and polarization among the most active raters, who disproportionately shape which notes surface, despite an explicit cross-partisan consensus design. Through counterfactual simulations that remove top raters, the study demonstrates that even tiny changes in participant composition can substantially alter note emergence and visibility, revealing structural instability. These findings imply that crowd-based fact-checking may concentrate epistemic power in a small, polarized minority, highlighting the need for robustness-aware designs to realize democratic moderation benefits.

Abstract

As platforms increasingly scale down professional fact-checking, community-based alternatives are promoted as more transparent and democratic. The main substitute being proposed is community-based contextualization, most notably Community Notes on X, where users write annotations and collectively rate their helpfulness under a consensus-oriented algorithm. This shift raises a basic empirical question: to what extent do users' social dynamics affect the emergence of Community Notes? We address this question by characterizing participation and political behavior, using the full public release of notes and ratings (between 2021 and 2025). We show that contribution activity is highly concentrated: a small minority of users accounts for a disproportionate share of ratings. Crucially, these high-activity contributors are not neutral volunteers: they are selective in the content they engage with and substantially more politically polarized than the overall contributor population. We replicate the notes' emergence process by integrating the open-source implementation of the Community Notes consensus algorithm used in production. This enables us to conduct counterfactual simulations that modify the display status of notes by varying the pool of raters. Our results reveal that the system is structurally unstable: the emergence and visibility of notes often depend on the behavior of a few dozen highly active users, and even minor perturbations in their participation can lead to markedly different outcomes. In sum, rather than decentralizing epistemic authority, community-based fact-checking on X reconfigures it, concentrating substantial power in the hands of a small, polarized group of highly active contributors.

Figures (6)

• Figure 1: Example of how a Community Notes labeled as Helpful is presented to all the users on X, including the contextual explanation and the associated sources displayed beneath the original tweet.
• Figure 2: Log-log distribution of ratings per contributor, with an inset showing the Lorenz curve. The heavy-tailed shape indicates that most users give few ratings, while a small minority is responsible for the majority of contributions: 20% of users account for roughly 80% of all ratings as displayed by the marker on the Lorenz curve.
• Figure 3: The plot shows the cumulative number of distinct authors rated as a function of the number of ratings provided by each rater. Observed data points are fitted with a saturation curve, indicating that raters concentrate their activity on a limited set of authors and reach the asymptotic number of engaged authors more quickly than expected under a random baseline obtained by reshuffling the author-rating couples.
• Figure 4: Schema of the evaluation pipeline. An author publishes a tweet and is classified as Democrat or Republican. A noter then produces a contextual note, labeled as MisleadingOrPotentiallyMisleading (M) or NotMisleading (NM). Finally, a rater evaluates the note by expressing agreement or disagreement, operationalized as Helpful or SomewhatHelpful (H/SH) versus NotHelpful (NH). Each rating is subsequently interpreted as a political signal (pro/anti Democrat or Republican) by jointly considering the author’s affiliation, the type of note, and the rater’s judgment.
• Figure 5: Distribution of political leaning $L$ across rating activity deciles. Negative (positive) values of $L$ indicate a preference toward rating notes associated with tweets by Democrat (Republican) accounts favorably. Polarization increases with user activity, indicating that the most active users tend to be the most politically polarized, and therefore engage in rating in a more biased manner.