Online Moderation in Competitive Action Games: How Intervention Affects Player Behaviors

TL;DR

The dual impact moderation has on reducing disruptive behavior and discouraging disruptive players from participating is uncovered, as well as differences in the effectiveness of quick and delayed moderation and the varying severity of punishment.

Abstract

Online competitive action games have flourished as a space for entertainment and social connections, yet they face challenges from a small percentage of players engaging in disruptive behaviors. This study delves into the under-explored realm of understanding the effects of moderation on player behavior within online gaming on an example of a popular title - Call of Duty(R): Modern Warfare(R)II. We employ a quasi-experimental design and causal inference techniques to examine the impact of moderation in a real-world industry-scale moderation system. We further delve into novel aspects around the impact of delayed moderation, as well as the severity of applied punishment. We examine these effects on a set of four disruptive behaviors including cheating, offensive user name, chat, and voice. Our findings uncover the dual impact moderation has on reducing disruptive behavior and discouraging disruptive players from participating. We further uncover differences in the effectiveness of quick and delayed moderation and the varying severity of punishment. Our examination of real-world gaming interactions sets a precedent in understanding the effectiveness of moderation and its impact on player behavior. Our insights offer actionable suggestions for the most promising avenues for improving real-world moderation practices, as well as the heterogeneous impact moderation has on indifferent players.

Figures (11)

• Figure 1: Distribution of Delay in Moderation; Figure showing both PDF and CDF of its distribution. Data generated from Feb 1st, 2023 to April 12th, 2023.
• Figure 2: Breakdown of the unique proportion of players by moderation reason in our dataset. Data generated from Feb 1st, 2023 to April 12th, 2023.
• Figure 3: Overview of the quasi-experimental setup used for answering our research questions. A) Represents a setup where we compare the players moderated immediately to the ones moderated with substantial delay. Crucially the comparison period here is the player behavior post-disruptive event. This means that for the "no-moderation" group (control) it reflects how players behave when effectively being unmoderated. B) Represents a setup where we investigate the impact of swiftness of moderation action on post moderation player behaviors. In contrast to the previous setup, here both groups are compared after being exposed to moderation actions. Crucially, we include only players who were eventually moderated by the human moderation team to addresses the known issue of noise in player-generated reports, where players may be unfairly reported for cheating or toxic behavior beres2021don.
• Figure 4: Analysis of the impact of moderation actions on repeated report rates (investigating RQ1), a measure of disruptive behavior. A) Effect of moderation/no-moderation (H1.1), quick moderation/delayed moderation (H1.2). B) Breakdown by action severity for moderation/no-moderation (H1.3) and quick moderation/delayed moderation (H1.4).
• Figure 5: Testing hypotheses associated with RQ2,impact of moderation actions on participation rates (proportion of days with matches played). A) Effect of moderation/no-moderation (H2.1), quick moderation/delayed moderation (H2.2). A) Breakdown by action severity (H2.3).