Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

More Than Just Warnings:Exploring the Ways of Communicating Credibility Assessment on Social Media

Huiyun Tang, Björn Rohles, Yuwei Chuai, Gabriele Lenzini, Anastasia Sergeeva

TL;DR

The study investigates how to convey AI-based credibility assessments for social media using content-based versus context-based indicators and binary versus fine-grained veracity signals. Through a preregistered online experiment (n = 537) with a 2x2 design, the authors show that both cue types aid users in evaluating post veracity, with fine-grained indicators enhancing information awareness and intention to use fact-checking tools. Context-based cues increase perceived informativeness relative to content-based cues, though overall trust and usefulness are broadly similar across indicator types. The results offer design guidance for user-friendly, customizable, two-layer explanations suitable for encrypted messaging environments, aiming to reduce misinformation spread through better-calibrated user judgment and critical engagement.

Abstract

Reducing the spread of misinformation is challenging. AI-based fact verification systems offer a promising solution by addressing the high costs and slow pace of traditional fact-checking. However, the problem of how to effectively communicate the results to users remains unsolved. Warning labels may seem an easy solution, but they fail to account for fuzzy misinformation that is not entirely fake. Additionally, users' limited attention spans and social media information should be taken into account while designing the presentation. The online experiment (n = 537) investigates the impact of sources and granularity on users' perception of information veracity and the system's usefulness and trustworthiness. Findings show that fine-grained indicators enhance nuanced opinions, information awareness, and the intention to use fact-checking systems. Source differences had minimal impact on opinions and perceptions, except for informativeness. Qualitative findings suggest the proposed indicators promote critical thinking. We discuss implications for designing concise, user-friendly AI fact-checking feedback.

More Than Just Warnings:Exploring the Ways of Communicating Credibility Assessment on Social Media

TL;DR

The study investigates how to convey AI-based credibility assessments for social media using content-based versus context-based indicators and binary versus fine-grained veracity signals. Through a preregistered online experiment (n = 537) with a 2x2 design, the authors show that both cue types aid users in evaluating post veracity, with fine-grained indicators enhancing information awareness and intention to use fact-checking tools. Context-based cues increase perceived informativeness relative to content-based cues, though overall trust and usefulness are broadly similar across indicator types. The results offer design guidance for user-friendly, customizable, two-layer explanations suitable for encrypted messaging environments, aiming to reduce misinformation spread through better-calibrated user judgment and critical engagement.

Abstract

Reducing the spread of misinformation is challenging. AI-based fact verification systems offer a promising solution by addressing the high costs and slow pace of traditional fact-checking. However, the problem of how to effectively communicate the results to users remains unsolved. Warning labels may seem an easy solution, but they fail to account for fuzzy misinformation that is not entirely fake. Additionally, users' limited attention spans and social media information should be taken into account while designing the presentation. The online experiment (n = 537) investigates the impact of sources and granularity on users' perception of information veracity and the system's usefulness and trustworthiness. Findings show that fine-grained indicators enhance nuanced opinions, information awareness, and the intention to use fact-checking systems. Source differences had minimal impact on opinions and perceptions, except for informativeness. Qualitative findings suggest the proposed indicators promote critical thinking. We discuss implications for designing concise, user-friendly AI fact-checking feedback.

Paper Structure

This paper contains 50 sections, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Communicating Content Veracity through Warning Labels
  4. Displaying System Opinions in Misinformation Correction Tasks
  5. Cognitive Processing and Evaluation of Information Credibility
  6. Content-based Credibility Assessment
  7. Content Verification through Facts Verification
  8. Characteristics of Writing Style
  9. Emotion of the Content
  10. Assessing Credibility Beyond the Content of Information (Context-based Sources of Credibility)
  11. Source Reputation and History of Information Endorser
  12. Bandwagon Characteristics of Information
  13. Research Questions
  14. Methodology
  15. Experimental Material
  16. ...and 35 more sections

Figures (4)

  • Figure 1: Experimental procedure
  • Figure 2: Example Interface Demonstrating the Integration of Fine-grained and content-based Indicators
  • Figure 3: Results of Users Ratings of Content-based vs. Context-based Credibility Indicators: Informative Competence (INFCOMP), Usefulness, Trustability, Trust in Information (TRUSTINF), and Frequency of Use (FREQUSE)
  • Figure 4: Results of Users Ratings of Binary and Fine-Grained Indicators: Informative Competence (INFCOMP), Usefulness, Trustability, Trust in Information (TRUSTINF), and Frequency of Use (FREQUSE)