Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Detecting Contextual Real-Time Toxicity for In-Game Chat

Zachary Yang, Nicolas Grenan-Godbout, Reihaneh Rabbany

TL;DR

The paper tackles real-time toxicity detection in in-game chat by introducing ToxBuster, a BERT-based token classifier that leverages full chat history and speaker metadata. It systematically evaluates ToxBuster across four datasets (R6S, FH, DOTA 2, CC), demonstrates superior performance over Cleanspeak, Perspective API, and Detoxify—especially when using chat history and speaker segmentation—and shows promising transferability between games and to news-comment domains. Ablation studies and per-class analyses highlight the value of contextual information for precision and recall, while post-game and proactive moderation assessments illustrate practical moderation benefits and workload reduction for human moderators. The work emphasizes the importance of up-to-date game-chat data, provides actionable insights for deployment, and outlines future directions including multilingual extension and debiasing.

Abstract

Real-time toxicity detection in online environments poses a significant challenge, due to the increasing prevalence of social media and gaming platforms. We introduce ToxBuster, a simple and scalable model that reliably detects toxic content in real-time for a line of chat by including chat history and metadata. ToxBuster consistently outperforms conventional toxicity models across popular multiplayer games, including Rainbow Six Siege, For Honor, and DOTA 2. We conduct an ablation study to assess the importance of each model component and explore ToxBuster's transferability across the datasets. Furthermore, we showcase ToxBuster's efficacy in post-game moderation, successfully flagging 82.1% of chat-reported players at a precision level of 90.0%. Additionally, we show how an additional 6% of unreported toxic players can be proactively moderated.

Towards Detecting Contextual Real-Time Toxicity for In-Game Chat

TL;DR

The paper tackles real-time toxicity detection in in-game chat by introducing ToxBuster, a BERT-based token classifier that leverages full chat history and speaker metadata. It systematically evaluates ToxBuster across four datasets (R6S, FH, DOTA 2, CC), demonstrates superior performance over Cleanspeak, Perspective API, and Detoxify—especially when using chat history and speaker segmentation—and shows promising transferability between games and to news-comment domains. Ablation studies and per-class analyses highlight the value of contextual information for precision and recall, while post-game and proactive moderation assessments illustrate practical moderation benefits and workload reduction for human moderators. The work emphasizes the importance of up-to-date game-chat data, provides actionable insights for deployment, and outlines future directions including multilingual extension and debiasing.

Abstract

Real-time toxicity detection in online environments poses a significant challenge, due to the increasing prevalence of social media and gaming platforms. We introduce ToxBuster, a simple and scalable model that reliably detects toxic content in real-time for a line of chat by including chat history and metadata. ToxBuster consistently outperforms conventional toxicity models across popular multiplayer games, including Rainbow Six Siege, For Honor, and DOTA 2. We conduct an ablation study to assess the importance of each model component and explore ToxBuster's transferability across the datasets. Furthermore, we showcase ToxBuster's efficacy in post-game moderation, successfully flagging 82.1% of chat-reported players at a precision level of 90.0%. Additionally, we show how an additional 6% of unreported toxic players can be proactively moderated.
Paper Structure (31 sections, 4 figures, 14 tables)

This paper contains 31 sections, 4 figures, 14 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methodology
  4. Dataset
  5. R6S & FH
  6. DOTA 2
  7. CC
  8. ToxBuster
  9. Chat History
  10. Chat Speaker Segmentation
  11. Baselines
  12. Cleanspeak
  13. Perspective API
  14. Detoxify
  15. Results and Discussions
  16. ...and 16 more sections

Figures (4)

  • Figure 1: ToxBuster with chat speaker segmentation. Input embeddings are the sum of the corresponding token, position, teamID, chat type and playerID. The chat history includes as many lines as possible.
  • Figure 2: ToxBuster Precision-Recall Curve on R6S. Average precision for non-toxic vs. toxic words is 95%.
  • Figure 3: Average number of flagged toxic lines/match over for increasingly chat-reported players. Players are often chat reported when having more than 5 flagged toxic lines.
  • Figure 4: ToxBuster Precision-Recall Curve per Toxic Class on R6S.