Social Bots: Detection and Challenges

TL;DR

The chapter addresses the threat of malicious social bots to online discourse and computational social science research. It surveys the literature with a scientometric lens, then uses Botometer as a case study to illustrate evolving detection methods, calibration, and generalizability challenges. The authors introduce advances from Botometer V3 to V4 and BotometerLite, including calibrated scores, Bayesian-inspired CAP estimates, and an ensemble of specialized classifiers to improve robustness across bot classes. A practical guide accompanies these developments, outlining actionable steps for researchers to detect and account for bots while acknowledging data-access constraints and the inevitable arms race with bot operators. Overall, the work highlights how detection tools must continually adapt to evolving bot tactics and platform policies to preserve research integrity and the reliability of online information ecosystems.

Abstract

While social media are a key source of data for computational social science, their ease of manipulation by malicious actors threatens the integrity of online information exchanges and their analysis. In this Chapter, we focus on malicious social bots, a prominent vehicle for such manipulation. We start by discussing recent studies about the presence and actions of social bots in various online discussions to show their real-world implications and the need for detection methods. Then we discuss the challenges of bot detection methods and use Botometer, a publicly available bot detection tool, as a case study to describe recent developments in this area. We close with a practical guide on how to handle social bots in social media research.

Figures (7)

• Figure 1: Numbers of publications regarding social bots (left) and publications citing them (right) over time. Publication types are shown in the insets.
• Figure 2: Network analyses of publications about social bots. Top: Citation network, with papers about social bots and citing publications represented as orange nodes and purple colors, respectively. Top-cited social bot papers are referred to with numbers and listed in Table \ref{['tab:papers']}. Bottom: Topical network, with node colors corresponding to communities of papers identified by modularity maximization.
• Figure 3: Timeline of Botometer.
• Figure 4: AUC scores of Random Forest classifiers trained on one dataset (row) and tested on another (column). Details about the datasets can be found in Table \ref{['table:dataset']}. Source: yang_scalable_2020.
• Figure 5: Bot score distributions for human (blue) and bot (red) accounts for Random Forest classifiers trained on the row datasets and tested on the column datasets. Source: sayyadiharikandeh2020detection.