A Survey on Automatic Credibility Assessment Using Textual Credibility Signals in the Era of Large Language Models

TL;DR

This survey addresses the automatic credibility assessment of online content from an NLP perspective in the era of Large Language Models. It consolidates 175 papers and introduces a unified taxonomy of textual credibility signals, with deep-dives into factuality, subjectivity/bias, and persuasion techniques, plus coverage of check-worthy and fact-checked claims and additional signals. It analyzes datasets, methods, and tools, and highlights open problems such as fragmentation, multilingual data scarcity, ethical considerations, and deployment challenges, while outlining opportunities for LLM-enabled, multi-signal, and multimodal approaches. The work aims to guide future research toward integrated, robust, and explainable credibility assessment that can support both automated detection and human-in-the-loop decision making.

Abstract

In the age of social media and generative AI, the ability to automatically assess the credibility of online content has become increasingly critical, complementing traditional approaches to false information detection. Credibility assessment relies on aggregating diverse credibility signals - small units of information, such as content subjectivity, bias, or a presence of persuasion techniques - into a final credibility label/score. However, current research in automatic credibility assessment and credibility signals detection remains highly fragmented, with many signals studied in isolation and lacking integration. Notably, there is a scarcity of approaches that detect and aggregate multiple credibility signals simultaneously. These challenges are further exacerbated by the absence of a comprehensive and up-to-date overview of research works that connects these research efforts under a common framework and identifies shared trends, challenges, and open problems. In this survey, we address this gap by presenting a systematic and comprehensive literature review of 175 research papers, focusing on textual credibility signals within the field of Natural Language Processing (NLP), which undergoes a rapid transformation due to advancements in Large Language Models (LLMs). While positioning the NLP research into the the broader multidisciplinary landscape, we examine both automatic credibility assessment methods as well as the detection of nine categories of credibility signals. We provide an in-depth analysis of three key categories: 1) factuality, subjectivity and bias, 2) persuasion techniques and logical fallacies, and 3) check-worthy and fact-checked claims. In addition to summarising existing methods, datasets, and tools, we outline future research direction and emerging opportunities, with particular attention to evolving challenges posed by generative AI.

Figures (8)

• Figure 1: Dimensions of credibility assessment. Bold font highlights such aspects which are within the scope of this survey (see Section \ref{['section:background-scope']} for more information). Please note that Content level under the dimension of Levels of credibility assessment and Content-based signals under the dimension of Taxonomies of credibility signals refer to two different concepts -- the former to the level of analysis, and the latter to the nature of the source data from which the credibility signals are detected.
• Figure 2: Unified categorization of textual credibility signals. Light blue nodes consist of credibility signals depicted in the existing taxonomies and categorizations. Bold font highlights categories of credibility signals on which this survey put a focus on (see Section \ref{['section:background-scope']} for more information).
• Figure 3: PRISMA 2020 flow diagram prisma2020diagram depicting the standardized methodology applied to collect the relevant research papers, together with a number of research papers processed in each step.
• Figure 4: Number of included papers according the direct (by keywords) and extended (by citations) search.
• Figure 5: Number of included papers according to the digital library grouped by each credibility signal category. Other refers to additional repositories, such as arXiv.