Evaluating and Addressing Fairness Across User Groups in Negative Sampling for Recommender Systems
Yueqing Xuan, Kacper Sokol, Mark Sanderson, Jeffrey Chan
TL;DR
The paper tackles fairness gaps in negative sampling for implicit-feedback recommenders, revealing that active users consistently receive more informative negatives and better recommendations. It systematically evaluates eight samplers across four datasets, showing that global increases in negative sampling ratio improve average performance but disproportionately benefit active users. To address this, it introduces a group-specific ratio framework optimized via Hyperband with informed priors, which yields notable gains for inactive users and overall accuracy. The work provides practical guidance for fairness-aware negative sampling and offers insights into cold-start implications and efficient hyperparameter search. These findings advance fair, data-balanced recommender design in real-world, imbalanced user bases.
Abstract
Recommender systems trained on implicit feedback data rely on negative sampling to distinguish positive items from negative items for each user. Since the majority of positive interactions come from a small group of active users, negative samplers are often impacted by data imbalance, leading them to choose more informative negatives for prominent users while providing less useful ones for users who are not so active. This leads to inactive users being further marginalised in the training process, thus receiving inferior recommendations. In this paper, we conduct a comprehensive empirical study demonstrating that state-of-the-art negative sampling strategies provide more accurate recommendations for active users than for inactive users. We also find that increasing the number of negative samples for each positive item improves the average performance, but the benefit is distributed unequally across user groups, with active users experiencing performance gain while inactive users suffering performance degradation. To address this, we propose a group-specific negative sampling strategy that assigns smaller negative ratios to inactive user groups and larger ratios to active groups. Experiments on eight negative samplers show that our approach improves user-side fairness and performance when compared to a uniform global ratio.