Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Modelling the Spread of New Information on Social Networks

Ziming Xu, Shi Zhou, Vasileios Lampos, Ingemar J. Cox

TL;DR

This paper tackles the challenge of predicting reposting of information on social networks when the topic is previously unseen, highlighting a critical out-of-distribution generalisation gap. It shows that theories and models that rely on message content alone underperform when faced with new hashtags, whereas incorporating user profile information and historical behavior markedly improves predictive accuracy, achieving substantial gains in F1 on OOD tasks. The authors introduce a comprehensive dataset of 14 trending hashtags with rich user and message features, and compare decision-tree, neural, and BERT-based approaches, finding that user-centric models can surpass text-based baselines, with DT-U/NN-U performing remarkably well in OOD settings. The work underscores the importance of evaluating both in-distribution and out-of-distribution performance for diffusion-related tasks and suggests that understanding who users are and how they have acted previously may be more predictive of reposting to unseen topics than the content itself. Practically, these results inform design choices for moderation, information diffusion modelling, and platform interventions by prioritising user history signals over raw message content for unseen-topic spread prediction.

Abstract

There has been considerable interest in modelling the spread of information on social networks using machine learning models. Here, we consider the problem of predicting the spread of new information, i.e. when a user propagates information about a topic previously unseen by the user. In existing work, information and users are randomly assigned to a test or training set, ensuring that both sets are drawn from the same distribution. In the spread of new information, the problem becomes an out-of-distribution generalisation classification task. Our experimental results reveal that while existing algorithms, which predominantly use features derived from the content of messages, perform well when the training and test distributions are the same, these algorithms perform much worse when the test set is out-of-distribution, i.e. when the topic (hashtag) of the testing data is absent from the training data. We then show that if the message features are supplemented or replaced with features derived from users' profile and past behaviour, the out-of-distribution prediction is greatly improved, with the F1 score increasing from 0.117 to 0.705. Our experimental results suggest that a significant component of reposting behaviour for previously unseen topics can be predicted from users' profile and past behaviour, and is largely content-agnostic.

Modelling the Spread of New Information on Social Networks

TL;DR

This paper tackles the challenge of predicting reposting of information on social networks when the topic is previously unseen, highlighting a critical out-of-distribution generalisation gap. It shows that theories and models that rely on message content alone underperform when faced with new hashtags, whereas incorporating user profile information and historical behavior markedly improves predictive accuracy, achieving substantial gains in F1 on OOD tasks. The authors introduce a comprehensive dataset of 14 trending hashtags with rich user and message features, and compare decision-tree, neural, and BERT-based approaches, finding that user-centric models can surpass text-based baselines, with DT-U/NN-U performing remarkably well in OOD settings. The work underscores the importance of evaluating both in-distribution and out-of-distribution performance for diffusion-related tasks and suggests that understanding who users are and how they have acted previously may be more predictive of reposting to unseen topics than the content itself. Practically, these results inform design choices for moderation, information diffusion modelling, and platform interventions by prioritising user history signals over raw message content for unseen-topic spread prediction.

Abstract

There has been considerable interest in modelling the spread of information on social networks using machine learning models. Here, we consider the problem of predicting the spread of new information, i.e. when a user propagates information about a topic previously unseen by the user. In existing work, information and users are randomly assigned to a test or training set, ensuring that both sets are drawn from the same distribution. In the spread of new information, the problem becomes an out-of-distribution generalisation classification task. Our experimental results reveal that while existing algorithms, which predominantly use features derived from the content of messages, perform well when the training and test distributions are the same, these algorithms perform much worse when the test set is out-of-distribution, i.e. when the topic (hashtag) of the testing data is absent from the training data. We then show that if the message features are supplemented or replaced with features derived from users' profile and past behaviour, the out-of-distribution prediction is greatly improved, with the F1 score increasing from 0.117 to 0.705. Our experimental results suggest that a significant component of reposting behaviour for previously unseen topics can be predicted from users' profile and past behaviour, and is largely content-agnostic.

Paper Structure

This paper contains 42 sections, 6 equations, 2 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Reposting on Social Media Platforms
  4. Reposting Prediction
  5. Existing Works
  6. Related Works
  7. Reposting Prediction
  8. Limitations of Previous Reposting Prediction
  9. Reliance on Message-Related Data
  10. Lack of Out-of-Distribution Evaluation
  11. Method
  12. X Data
  13. Message Data and Features
  14. User Data and Features
  15. User Profile Features (U-P)
  16. ...and 27 more sections

Figures (2)

  • Figure 1: Neural network (NN) models based on different types of data.
  • Figure 2: In-distribution and out-of-distribution predictions by Decision Tree (DT) models using different types of features. User-related features U includes U-P, U-HA and U-HM.