Quantifying Feature Importance for Online Content Moderation

TL;DR

The paper tackles understanding how users respond to moderation interventions by reframing the problem as ordinal quantification over population-level changes in activity, toxicity, and diversity, using 753 features organized into 9 groups. It introduces a greedy feature-block selection approach and demonstrates that a small, consistent core of features is informative across tasks while many features are task-specific; predictive performance varies by task, with activity and toxicity being easier to estimate than diversity. By benchmarking quantification methods and applying EMQ with optimized feature subsets (EMQO), the study achieves measurable reductions in quantification error (e.g., 17.20% for activity, 23.76% for toxicity, 18.15% for diversity) and reveals substantial, task-specific feature dependencies. The findings highlight the complexity of post-moderation behavior and advocate for objective- and task-specific moderation strategies that leverage data-driven signals about users and interventions to improve safety and effectiveness.

Abstract

Accurately estimating how users respond to moderation interventions is paramount for developing effective and user-centred moderation strategies. However, this requires a clear understanding of which user characteristics are associated with different behavioural responses, which is the goal of this work. We investigate the informativeness of 753 socio-behavioural, linguistic, relational, and psychological features, in predicting the behavioural changes of 16.8K users affected by a major moderation intervention on Reddit. To reach this goal, we frame the problem in terms of "quantification", a task well-suited to estimating shifts in aggregate user behaviour. We then apply a greedy feature selection strategy with the double goal of (i) identifying the features that are most predictive of changes in user activity, toxicity, and participation diversity, and (ii) estimating their importance. Our results allow identifying a small set of features that are consistently informative across all tasks, and determining that many others are either task-specific or of limited utility altogether. We also find that predictive performance varies according to the task, with changes in activity and toxicity being easier to estimate than changes in diversity. Overall, our results pave the way for the development of accurate systems that predict user reactions to moderation interventions. Furthermore, our findings highlight the complexity of post-moderation user behaviour, and indicate that effective moderation should be tailored not only to user traits but also to the specific objective of the intervention.

Figures (4)

• Figure 1: Effects distribution and class distribution as a function of the parameter $\Delta$ for each of the considered tasks: activity, diversity, and toxicity. For the effect scores, we report the kernel density estimation (KDE) of the continuous values. The red dashed lines indicate the thresholds $\pm \Delta_{\text{Moderate}}$, while the red solid lines indicate $\pm \Delta_{\text{High}}$.
• Figure 2: Quantification error of the models as a function of training set size. The curves report the error trends across incremental subsets for the activity, toxicity, and diversity tasks.
• Figure 3: Percentage $\text{RIE}_{S}^T$ for each subgroup across each task. Darker green-coloured cells indicate higher importance of a subgroup for a given task, whereas blue-coloured cells denote that the subgroup was not used in that task.
• Figure 4: Pairwise similarity between the sets of features selected for the three behavioural prediction tasks: activity, toxicity, and diversity. Results are shown as heatmaps of the Jaccard index (left) and Rank-Biased Overlap (right). Larger values indicate greater similarity. Both measures reveal overall low similarity, highlighting that each task relies on largely distinct sets of predictive features.