Identifying and Upweighting Power-Niche Users to Mitigate Popularity Bias in Recommendations

TL;DR

The paper tackles popularity bias in recommender systems by identifying a distinct power-niche user segment—high-activity users who prefer niche items—and demonstrating its prevalence and value. It introduces PAIR, a Bayesian Personalized Ranking reweighting framework that jointly weights by user activity level and item popularity via the hyperparameters $\alpha$ and $\beta$, effectively upweighting power-niche interactions. Across Gowalla, Yelp2018, and Amazon-Book, PAIR achieves Pareto-dominant improvements over vanilla BPR and several baselines, boosting Recall@20, Precision@20, and NDCG@20 while reducing Popularity-Opportunity Bias. The findings indicate that incorporating both activity and popularity heterogeneity yields substantial gains for both niche and mainstream performance, offering a practical pathway to mitigate popularity bias without sacrificing overall recommendation quality.

Abstract

Recommender systems have been shown to exhibit popularity bias by over-recommending popular items and under-recommending relevant niche items. We seek to understand niche users in benchmark recommendation datasets as a step toward mitigating popularity bias. We find that, compared to mainstream users, niche-preferring users exhibit a longer-tailed activity-level distribution, indicating the existence of users who both prefer niche items and exhibit high activity levels on platforms. We partition users along two axes: (1) activity level ("power" vs. "light") and (2) item-popularity preference ("mainstream" vs. "niche"), and show that in three benchmark datasets, the number of power-niche users (high activity and niche preference) is statistically significantly larger than expected. We also find that interaction data from power-niche users is especially valuable for improving recommendations for not only niche but also mainstream users. In contrast, many existing popularity bias mitigation methods have focused on upweighting niche users regardless of activity level. Motivated by the value of power-niche user data, we propose PAIR (Popularity-and-Activity-Informed Reweighting), a framework for reweighting the Bayesian Personalized Ranking (BPR) loss that simultaneously reweights based on user activity level and item popularity, upweighting power-niche users the most. We instantiate the framework on both deep and shallow collaborative filtering models, and experiments on benchmark datasets show that PAIR reduces popularity bias and can increase overall performance. Although existing popularity-bias mitigation methods yield a trade-off between performance and bias, our results suggest that considering both user activity level and popularity preference leads to Pareto-dominant performance.

Figures (7)

• Figure 1: Distribution of user activity levels for mainstream and niche users in the Gowalla, Yelp2018, and Amazon-Book datasets. Niche users have below-median item-popularity preference. In each dataset, the distribution of activity levels for niche users is longer (further right) than the distribution for mainstream users, indicating the existence of users who prefer niche items and exhibit high activity levels.
• Figure 2: Item interaction patterns of different user groups, compared to a configuration null model. Users are grouped by activity-level and item-popularity quantiles. Top row: we show the distribution of power-niche representation (expressed as a percentage) under the null model, as well as the same quantity observed in the data. Bottom row: we show the relative difference in percentage representation for the observed data and the null model for all user groups. For each dataset, the number of power-niche users---high activity level and niche item popularity preference---is higher than expected under the configuration null model.
• Figure 3: An overview of our method for valuing data from each user quadrant. A group of users is uniformly sampled at random and labeled the "fixed set". Two models are then trained: one using only training data from the fixed-set users and another that includes an additional sample from the target quadrant. The pink box under the treatment training data is representative of a sample---mutually exclusive of the fixed set---of power-niche users. Both models are evaluated on the fixed-set users. The procedure is repeated for multiple trials, and the value of a quadrant is the average change in performance. We evaluate the value of quadrant data at multiple treatment ratios, which is the relative size of the additional treatment data relative to the fixed set.
• Figure 4: To inform our reweighting, we examine the value of each quadrant for improving niche recommendation performance (top) and overall (bottom) performance. We measure the value of a quadrant by the relative change in performance (Recall@$k$) for all users and niche users when the model is re-trained after users from a specified quadrant are added to the training dataset. The test set is fixed. Power-niche users are not only valuable for improving niche user performance, but are also the most valuable quadrant for improving overall performance.
• Figure 5: Under PAIR, our reweighting framework, power-niche users are upweighted the most relative to the BPR loss. The weight of user $u$ is the number of positive items sampled per epoch. The percentages in the heatmaps indicate the change in the relative total weight of each quadrant when applying the optimized hyperparameters for PAIR to MF and LGN; hence, the percentages sum to zero in each heatmap. In contrast to power-niche users, light users exhibit a decrease in weight, and power-mainstream users exhibit a marginal increase in weight.