Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multilevel User Credibility Assessment in Social Networks

Mohammad Moradi, Mostafa Haghir Chehreghani

TL;DR

The paper tackles multilevel user credibility assessment on social networks by addressing the lack of fine-grained datasets and proposing MultiCred, a multimodal framework that fuses profile, textual, and interaction signals to classify users into multiple credibility tiers. It introduces a Twitter-based dataset labeled via NewsGuard, employs deep language models for textual analysis, autoencoders for embedding compression, and a neural classifier to predict several credibility levels, with SMOTE addressing class imbalance. Empirical results show MultiCred consistently outperforms two strong baselines across 4-, 6-, 8-, and 10-class setups, with SHAP analyses highlighting the dominant role of profile features and the additive value of text-derived representations. The work offers a nuanced approach to credibility that supports more effective moderation and trust-aware information retrieval, while outlining avenues for incorporating images and social-graph structure in future work.

Abstract

Online social networks serve as major platforms for disseminating both real and fake news. Many users--intentionally or unintentionally--spread harmful content, misinformation, and rumors in domains such as politics and business. Consequently, user credibility assessment has become a prominent area of research in recent years. Most existing methods suffer from two key limitations. First, they treat credibility as a binary task, labeling users as either genuine or fake, whereas real-world applications often demand a more nuanced, multilevel evaluation. Second, they rely on only a subset of relevant features, which constrains their predictive performance. In this paper, we address the lack of a dataset suitable for multilevel credibility assessment by first devising a collection method tailored to this task. We then propose the \textit{MultiCred} model, which assigns users to one of several credibility tiers based on a rich and diverse set of features extracted from their profiles, tweets, and comments. MultiCred leverages deep language models for textual analysis and deep neural networks for non-textual data processing. Our extensive experiments demonstrate that MultiCred significantly outperforms existing approaches across multiple accuracy metrics. Our code is publicly available at https://github.com/Mohammad-Moradi/MultiCred.

Multilevel User Credibility Assessment in Social Networks

TL;DR

The paper tackles multilevel user credibility assessment on social networks by addressing the lack of fine-grained datasets and proposing MultiCred, a multimodal framework that fuses profile, textual, and interaction signals to classify users into multiple credibility tiers. It introduces a Twitter-based dataset labeled via NewsGuard, employs deep language models for textual analysis, autoencoders for embedding compression, and a neural classifier to predict several credibility levels, with SMOTE addressing class imbalance. Empirical results show MultiCred consistently outperforms two strong baselines across 4-, 6-, 8-, and 10-class setups, with SHAP analyses highlighting the dominant role of profile features and the additive value of text-derived representations. The work offers a nuanced approach to credibility that supports more effective moderation and trust-aware information retrieval, while outlining avenues for incorporating images and social-graph structure in future work.

Abstract

Online social networks serve as major platforms for disseminating both real and fake news. Many users--intentionally or unintentionally--spread harmful content, misinformation, and rumors in domains such as politics and business. Consequently, user credibility assessment has become a prominent area of research in recent years. Most existing methods suffer from two key limitations. First, they treat credibility as a binary task, labeling users as either genuine or fake, whereas real-world applications often demand a more nuanced, multilevel evaluation. Second, they rely on only a subset of relevant features, which constrains their predictive performance. In this paper, we address the lack of a dataset suitable for multilevel credibility assessment by first devising a collection method tailored to this task. We then propose the \textit{MultiCred} model, which assigns users to one of several credibility tiers based on a rich and diverse set of features extracted from their profiles, tweets, and comments. MultiCred leverages deep language models for textual analysis and deep neural networks for non-textual data processing. Our extensive experiments demonstrate that MultiCred significantly outperforms existing approaches across multiple accuracy metrics. Our code is publicly available at https://github.com/Mohammad-Moradi/MultiCred.
Paper Structure (24 sections, 8 equations, 5 figures, 11 tables, 1 algorithm)

This paper contains 24 sections, 8 equations, 5 figures, 11 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Methods based on user profile features
  4. Methods based on textual features
  5. Methods based on both textual and non-textual features
  6. Preliminaries
  7. Dataset
  8. Existing datasets
  9. Limitations of existing datasets
  10. Our data collection method
  11. Our proposed method
  12. Data analysis and feature selection
  13. Non-textual features
  14. Textual features
  15. Aggregation
  16. ...and 9 more sections

Figures (5)

  • Figure 1: High-level architecture of MultiCred.
  • Figure 2: The final embedding created for each user.
  • Figure 3: Overall importance of feature categories measured by summed absolute SHAP values.
  • Figure 4: Importance of individual profile features per credibility class measured by summed absolute SHAP values.
  • Figure 5: Accuracy comparison of MultiCred and two baseline methods across varying numbers of classes.