How Do Recommendation Models Amplify Popularity Bias? An Analysis from the Spectral Perspective

TL;DR

This work investigates why recommendation models amplify popularity bias when trained on long-tailed data. Using a spectral perspective, it reveals that item popularity is memorized in the principal spectrum of the predicted score matrix $\hat{\mathbf{Y}}$ and that dimension reduction magnifies this effect. The authors propose ReSN, a spectral-norm regularizer that directly penalizes the magnitude of the principal singular value with efficient approximations leveraging the low-rank structure $\mathbf{U}\mathbf{V}^{\top}$ and the popularity vector $\mathbf{r}$. The approach is supported by theoretical bounds and extensive experiments on seven real-world datasets showing improved fairness-accuracy trade-offs with minimal computational overhead. Overall, ReSN provides a scalable, model-agnostic tool for mitigating popularity bias in embedding-based recommender systems by targeting the dominant spectral component of predictions.

Abstract

Recommendation Systems (RS) are often plagued by popularity bias. When training a recommendation model on a typically long-tailed dataset, the model tends to not only inherit this bias but often exacerbate it, resulting in over-representation of popular items in the recommendation lists. This study conducts comprehensive empirical and theoretical analyses to expose the root causes of this phenomenon, yielding two core insights: 1) Item popularity is memorized in the principal spectrum of the score matrix predicted by the recommendation model; 2) The dimension collapse phenomenon amplifies the relative prominence of the principal spectrum, thereby intensifying the popularity bias. Building on these insights, we propose a novel debiasing strategy that leverages a spectral norm regularizer to penalize the magnitude of the principal singular value. We have developed an efficient algorithm to expedite the calculation of the spectral norm by exploiting the spectral property of the score matrix. Extensive experiments across seven real-world datasets and three testing paradigms have been conducted to validate the superiority of the proposed method.

Figures (7)

• Figure 1: Illustration of two core insights.
• Figure 2: Illustration of how dimension reduction impacts popularity bias in Movielens: (a)-(b) the proportion of popular items in recommendations and the ratio of the largest singular value ($\sigma_1^2/\sum_{1\le k\le L} {\sigma_k^2}$) with varying embedding dimensions and training epochs, respectively; (c) how singular values evolves during training.
• Figure 3: Pareto curves of compared methods illustrating the trade-off between accuracy and fairness under the common testing paradigm.
• Figure 4: The proportion of popular items in recommendations and the ratio of the largest singular value ($\sigma_1^2/\sum_{1\le k\le L} {\sigma_k^2}$) and NDCG@20 with varying $\beta$.
• Figure 5: Long-tailed distribution of item popularity in recommendation datasets.

Theorems & Definitions (3)

• Theorem 1: Popularity Memorization Effect
• Theorem 2: Popularity bias amplification
• Theorem 3: Trajectory of singular values (Eq. (6) in saxe2019mathematical)