Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Is Fact-Checking Politically Neutral? Asymmetries in How U.S. Fact-Checking Organizations Pick Up False Statements Mentioning Political Elites

Yuwei Chuai, Jichang Zhao, Nicolas Pröllochs, Gabriele Lenzini

Abstract

Political elites play an important role in the proliferation of online misinformation. However, an understanding of how fact-checking platforms pick up politicized misinformation for fact-checking is still in its infancy. Here, we conduct an empirical analysis of mentions of U.S. political elites within fact-checked statements. For this purpose, we collect a comprehensive dataset consisting of 35,014 true and false statements that have been fact-checked by two major fact-checking organizations (Snopes, PolitiFact) in the U.S. between 2008 and 2023, i.e., within an observation period of 15 years. Subsequently, we perform content analysis and explanatory regression modeling to analyze how veracity is linked to mentions of U.S. political elites in fact-checked statements. Our analysis yields the following main findings: (i) Fact-checked false statements are, on average, 20% more likely to mention political elites than true fact-checked statements. (ii) There is a partisan asymmetry such that fact-checked false statements are 88.1% more likely to mention Democrats, but 26.5% less likely to mention Republicans, compared to fact-checked true statements. (iii) Mentions of political elites in fact-checked false statements reach the highest level during the months preceding elections. (iv) Fact-checked false statements that mention political elites carry stronger other-condemning emotions and are more likely to be pro-Republican, compared to fact-checked true statements. In sum, our study offers new insights into understanding mentions of political elites in false statements on U.S. fact-checking platforms, and bridges important findings at the intersection between misinformation and politicization.

Is Fact-Checking Politically Neutral? Asymmetries in How U.S. Fact-Checking Organizations Pick Up False Statements Mentioning Political Elites

Abstract

Political elites play an important role in the proliferation of online misinformation. However, an understanding of how fact-checking platforms pick up politicized misinformation for fact-checking is still in its infancy. Here, we conduct an empirical analysis of mentions of U.S. political elites within fact-checked statements. For this purpose, we collect a comprehensive dataset consisting of 35,014 true and false statements that have been fact-checked by two major fact-checking organizations (Snopes, PolitiFact) in the U.S. between 2008 and 2023, i.e., within an observation period of 15 years. Subsequently, we perform content analysis and explanatory regression modeling to analyze how veracity is linked to mentions of U.S. political elites in fact-checked statements. Our analysis yields the following main findings: (i) Fact-checked false statements are, on average, 20% more likely to mention political elites than true fact-checked statements. (ii) There is a partisan asymmetry such that fact-checked false statements are 88.1% more likely to mention Democrats, but 26.5% less likely to mention Republicans, compared to fact-checked true statements. (iii) Mentions of political elites in fact-checked false statements reach the highest level during the months preceding elections. (iv) Fact-checked false statements that mention political elites carry stronger other-condemning emotions and are more likely to be pro-Republican, compared to fact-checked true statements. In sum, our study offers new insights into understanding mentions of political elites in false statements on U.S. fact-checking platforms, and bridges important findings at the intersection between misinformation and politicization.
Paper Structure (29 sections, 2 equations, 5 figures, 25 tables)

This paper contains 29 sections, 2 equations, 5 figures, 25 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Research Questions
  4. Data and Methods
  5. Data Source: Fact-Checked Statements
  6. Politicization
  7. Other-Condemning Emotions
  8. Explanatory Regression Models
  9. Empirical Analysis
  10. Mentions of Political Elites (RQ1 & RQ2)
  11. Elections (RQ3)
  12. Other-Condemning Emotions (RQ4)
  13. Exploratory Analysis & Robustness Checks
  14. Discussion
  15. Conclusion
  16. ...and 14 more sections

Figures (5)

  • Figure 1: The five most frequently mentioned politicians in fact-checked statements.
  • Figure 2: Dataset overview. \ref{['fig:claim_count_by_time']} The monthly count of fact-checked true and false statements in our observation period between 2008 and 2023. The eight grey vertical dash lines correspond to eight U.S. election dates. \ref{['fig:months_elections_ccdf']} Complementary Cumulative Distribution Functions (CCDFs) for $\mathit{MTE}$ (months to elections) in fact-checked true and false statements. The difference in distributions is statistically significant according to a KS-test ($\mathit{KS} = 0.073, p < 0.001$). \ref{['fig:months_elections_politician_ccdf']} CCDFs for $\mathit{MTE}$ in fact-checked statements w/ and w/o mentions of political elites. The difference in distributions is statistically significant according to a KS-test ($\mathit{KS} =0.151, p < 0.001$).
  • Figure 3: The politicization ratios of the nine topics that have statistically significant differences during Obama's and Trump's incumbencies. The Chi-Square test is conducted for each topic group. *$^{*}$$p<0.05$, **$^{**}$$p<0.01$, ***$^{***}$$p<0.001$.
  • Figure 4: Predicted marginal means for the probabilities of mentions of \ref{['fig:politician_margin']} political elites of either political party, \ref{['fig:republican_margin']} Republicans, and \ref{['fig:democrat_margin']} Democrats in fact-checked true and false statements. The error bands represent 95% Confidence Intervals (CIs).
  • Figure S1: Topics in fact-checked statements. \ref{['fig:issue_weight_reg_fit']} The distribution of the differences in shared topics between fact-checked true and false statements. The embedded scatters describe the ratios of each shared topic in fact-checked true and false statements. \ref{['fig:topic_diff_ratio']} The ratios of statement groups in which the mentions of political elites are significantly associated with the veracity of fact-checked statements under different thresholds of group size. \ref{['fig:topic_diff_errorbar']} The differences in politicization ratios between fact-checked true and false statements under different thresholds of group size. The error bars represent 95% Confidence Intervals (CIs).