Reinforcement Learning for Efficient Toxicity Detection in Competitive Online Video Games
Jacob Morrier, Rafal Kocielnik, R. Michael Alvarez
TL;DR
The paper addresses efficient toxicity monitoring in competitive online games by casting it as a contextual bandit problem that decides at match start which players to monitor. It introduces a LinUCB-based algorithm that uses domain-informed covariates to predict toxicity likelihood, balancing exploration and exploitation with daily parameter updates. Using Call of Duty: MWIII data and ToxMod signals, the approach outperforms baseline explore–then–commit strategies, achieving substantial gains in detection rate for a fixed monitoring budget. The findings suggest toxicity is context-dependent and that monitoring policies guided by contextual cues can improve safety and user experience, with practical implications for production deployments and future research directions.
Abstract
Online platforms take proactive measures to detect and address undesirable behavior, aiming to focus these resource-intensive efforts where such behavior is most prevalent. This article considers the problem of efficient sampling for toxicity detection in competitive online video games. To make optimal monitoring decisions, video game service operators need estimates of the likelihood of toxic behavior. If no model is available for these predictions, one must be estimated in real time. To close this gap, we propose a contextual bandit algorithm that makes monitoring decisions based on a small set of variables that, according to domain expertise, are associated with toxic behavior. This algorithm balances exploration and exploitation to optimize long-term outcomes and is deliberately designed for easy deployment in production. Using data from the popular first-person action game Call of Duty: Modern Warfare III, we show that our algorithm consistently outperforms baseline algorithms that rely solely on players' past behavior. This finding has substantive implications for the nature of toxicity. It also illustrates how domain expertise can be harnessed to help video game service operators identify and mitigate toxicity, ultimately fostering a safer and more enjoyable gaming experience.