Fairness-Aware Multi-Group Target Detection in Online Discussion

TL;DR

This work addresses target-group detection in online discourse as a multi-label task where posts can target multiple demographic groups, and fairness across groups is crucial. It introduces Accuracy Parity (AP) as the fairness objective and proposes GAP and its multi-target extension GAP_{multi} to directly optimize balanced group performance while preserving overall accuracy. Theoretical results show EO and AP cannot be satisfied simultaneously under unequal base rates, motivating AP as a more appropriate fairness criterion for this setting. Empirically, GAP_{multi} yields improved fairness (smaller pairwise BA disparities and higher macro BA) with competitive utility across two large datasets (MHS and HateXplain), and the approach scales with parallel GPU computation. The work provides a practical, reproducible framework for fair target-group detection in toxic content analysis and related moderation tasks, with ethical considerations and deployment guidance.

Abstract

Target-group detection is the task of detecting which group(s) a piece of content is ``directed at or about''. Applications include targeted marketing, content recommendation, and group-specific content assessment. Key challenges include: 1) that a single post may target multiple groups; and 2) ensuring consistent detection accuracy across groups for fairness. In this work, we investigate fairness implications of target-group detection in the context of toxicity detection, where the perceived harm of a social media post often depends on which group(s) it targets. Because toxicity is highly contextual, language that appears benign in general can be harmful when targeting specific demographic groups. We show our {\em fairness-aware multi-group target detection} approach both reduces bias across groups and shows strong predictive performance, surpassing existing fairness-aware baselines. To enable reproducibility and spur future work, we share our code online.

Figures (10)

• Figure 1: Our multi-group target detection architecture. The model has shared parameters to learn both general and group-targeted language properties. The classification nodes each learn group-specific mappings.
• Figure 2: Visualization of the BA values achieved by each loss over the 7 demographic groups in MHS corpus. The maximum difference (Max. Diff.) between the maximum and minimum BA achieved for each loss across groups is also shown. $GAP_{multi}$ performs best with lowest Max Diff. of 5.5, showcasing that it greatly reduces the disparities in performance across groups vs. other losses.
• Figure 3: Heatmap of pairwise absolute difference of BA across groups in test set (MHS corpus) as an indicator for bias and disparate impact. OE has the highest performance gap (Max Diff = 21.9) across groups as indicated by the extremes of color, not only across one group- pair but consistently across multiple group pairs. $GAP_{multi}$ has the least spread in pairwise error values (Max. Diff. = 5.5), evident from the flatness of color, indicating least disparate impact across groups.
• Figure 4: Our $GAP_{multi}$ formulation enables parallel GPU computation of pairwise-group errors, achieving constant $\mathcal{O}(1)$ scaling complexity. Both the Optimized and Naive curves are for $GAP_{multi}$ loss with/-out the parallelism involved. The serial bottleneck in the Naive version leads to a quadratic scaling in runtime w.r.t. group size. For our implementation, the Optimized version has runtime per epoch with only constant increase, even when scaling from 2 to 30 demographic groups.
• Figure 5: Summary statistics of the MHS corpus sachdeva2022assessing show the distribution of posts targeting demographic groups. The Black community is the statistical majority, while Native American and Pacific Islander are statistical minorities. Additionally, the dataset includes posts targeting multiple groups, reflecting its multi-group nature.

Theorems & Definitions (3)

• Theorem 1
• Theorem 2
• Remark