The Chilling: Identifying Strategic Antisocial Behavior Online and Examining the Impact on Journalists

TL;DR

This paper addresses the problem of identifying strategic antisocial behavior online, particularly coordinated attacks against female journalists on Twitter. It introduces ConvTreeTrans, a tree-structured Transformer that leverages conversation hierarchies to classify replies into attackers, supporters, and bystanders and to infer latent strategies via a joint loss framework. The authors show ConvTreeTrans outperforms baselines in both group classification and strategy discovery, reveal that attacker presence correlates with chilling effects on journalists’ posting behavior, and illuminate distinct strategic patterns across groups. The findings highlight practical implications for platform moderation and journalist-facing tools to curb coordinated toxicity and foster healthier online conversations. The work advances understanding of how structure in online conversations relates to strategic behavior and contributes actionable insights for real-time toxicity detection and moderation design.

Abstract

On social platforms like Twitter, strategic targeted attacks are becoming increasingly common, especially against vulnerable groups such as female journalists. Two key challenges in identifying strategic online behavior are the complex structure of online conversations and the hidden nature of potential strategies that drive user behavior. To address these, we develop a new tree structured Transformer model that categorizes replies based on their hierarchical conversation structures. Extensive experiments demonstrate that our proposed classification model can effectively detect different user groups, namely attackers, supporters, and bystanders, and their latent strategies. To demonstrate the utility of our approach, we apply this classifier to real time Twitter data and conduct a series of quantitative analyses on the interactions between journalists with different groups of users. Our classification approach allows us to not only explore strategic behaviors of attackers but also those of supporters and bystanders who engage in online interactions. When examining the impact of online attacks, we find a strong correlation between the presence of attackers' interactions and chilling effects, where journalists tend to slow their subsequent posting behavior. This paper provides a deeper understanding of how different user groups engage in online discussions and highlights the detrimental effects of attacker presence on journalists, other users, and conversational outcomes. Our findings underscore the need for social platforms to develop tools that address coordinated toxicity. By detecting patterns of coordinated attacks early, platforms could limit the visibility of toxic content to prevent escalation. Additionally, providing journalists and users with tools for real time reporting could empower them to manage hostile interactions more effectively.

Figures (45)

• Figure 1: An example conversation tree. The root node denotes a post created by a journalist; the remaining nodes signify replies or subsequent replies to initial responses. Red nodes denote attackers towards the journalist, blue nodes denote supporters, and green nodes represent bystanders. See \ref{['sec:1']}
• Figure 2: An example of the journalist's activity timeline. The blue line represents the number of the journalist posting per month and the red line represents the number of replies received by the journalist per month.
• Figure 3: Our framework. $\bigoplus$ denotes concatenation, $x_i, p_i, r_{ij}$ denote the metadata, global path and local path for node $i$. The metadata includes features such as language, creation time, number of retweets, replies, likes, views, toxic scores, topics, and the presence of URLs. Global path includes the path from the root to node $i$, and the local path refers to the path between the reply $t_i$ and its parent node $j$. $z_i$ gives the output of the Transformer model, $L_c$ and $L_s$ are loss for the classification and strategy discovery modules.
• Figure 4: User clusters. For these three journalists, we see clear separations between attackers (red) and supporters (blue), but there are overlaps between bystanders (green) and attackers (red), indicating their similarity in behaviors. (See \ref{['sec:5.6']}).
• Figure 5: Distributions of strategic behaviors among attackers, supporters, and bystanders with respect to the number of strategies (See \ref{['sec:6.1']}). Attackers seem to have clear strategies (strategies $s_1$ and $s_2$), often targeting users with similar interests or engaging in discussions on related topics while supporters do not appear to follow a specific strategy, showing a more varied or less coordinated approach to their replies. This suggests that attackers may operate in a more organized manner, while supporters exhibit more spontaneous or less structured behavior.