Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unveiling News Publishers Trustworthiness Through Social Interactions

Manuel Pratelli, Fabio Saracco, Marinella Petrocchi

TL;DR

The paper tackles scalable assessment of online news publishers' trustworthiness by analyzing social-media interactions rather than manual content analysis. It introduces URL NECs built from a maximum-entropy null model and community detection, identifies Discussion Supporters as voters, and uses their aggregated judgments to label publishers with two classes (trustworthy vs untrustworthy). External labels from NewsGuard guide the evaluation; four voter-strategy variants balance coverage and labeling cost, enabling automatic estimation for previously unclassified outlets. Results show substantial publisher coverage (roughly 70-90% across trust categories) and competitive classification performance (balanced accuracy up to around 0.81 in data-rich settings), with potential applicability to other social platforms and domains.

Abstract

With the primary goal of raising readers' awareness of misinformation phenomena, extensive efforts have been made by both academic institutions and independent organizations to develop methodologies for assessing the trustworthiness of online news publishers. Unfortunately, existing approaches are costly and face critical scalability challenges. This study presents a novel framework for assessing the trustworthiness of online news publishers using user interactions on social media platforms. The proposed methodology provides a versatile solution that serves the dual purpose of i) identifying verifiable online publishers and ii) automatically performing an initial estimation of the trustworthiness of previously unclassified online news outlets.

Unveiling News Publishers Trustworthiness Through Social Interactions

TL;DR

The paper tackles scalable assessment of online news publishers' trustworthiness by analyzing social-media interactions rather than manual content analysis. It introduces URL NECs built from a maximum-entropy null model and community detection, identifies Discussion Supporters as voters, and uses their aggregated judgments to label publishers with two classes (trustworthy vs untrustworthy). External labels from NewsGuard guide the evaluation; four voter-strategy variants balance coverage and labeling cost, enabling automatic estimation for previously unclassified outlets. Results show substantial publisher coverage (roughly 70-90% across trust categories) and competitive classification performance (balanced accuracy up to around 0.81 in data-rich settings), with potential applicability to other social platforms and domains.

Abstract

With the primary goal of raising readers' awareness of misinformation phenomena, extensive efforts have been made by both academic institutions and independent organizations to develop methodologies for assessing the trustworthiness of online news publishers. Unfortunately, existing approaches are costly and face critical scalability challenges. This study presents a novel framework for assessing the trustworthiness of online news publishers using user interactions on social media platforms. The proposed methodology provides a versatile solution that serves the dual purpose of i) identifying verifiable online publishers and ii) automatically performing an initial estimation of the trustworthiness of previously unclassified online news outlets.
Paper Structure (15 sections, 3 equations, 6 figures, 4 tables)

This paper contains 15 sections, 3 equations, 6 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Useful definitions
  3. Methods
  4. Data collection
  5. Selection of voters through URL NEC
  6. Endogenous characterization of voters
  7. Classification of online publishers via voters
  8. Results
  9. Coverage of Publishers
  10. Classification Performance
  11. Discussion
  12. Related Work
  13. Conclusions
  14. Purity measure
  15. Online Resources

Figures (6)

  • Figure 1: Schematization of the procedure for classifying the online publisher trustworthiness
  • Figure 2: Purity levels of URL NECs. Trusted URLs on the left, untrusted URLs on the right
  • Figure 3: Number of voters wrt the minimum number of shared publishers and the adopted strategy
  • Figure 4: How many publishers can we reach given the original number of publishers, the set of voters, and the minimum number of publishers shared by the voters?
  • Figure 5: Publisher classification performance
  • ...and 1 more figures