Break Out of a Pigeonhole: A Unified Framework for Examining Miscalibration, Bias, and Stereotype in Recommender Systems

TL;DR

This work tackles miscalibration, bias, and stereotyping in recommender systems by proposing a unified framework that decomposes miscalibration into bias and variance and introduces system-induced effects such as stereotype and inflated diversity. Using MovieLens 1M and five algorithms, the authors show that complex models achieve better item-level accuracy but worse category-level calibration, while simpler models exaggerate stereotypes; miscalibration and biases disproportionately affect women, older users, and atypical users. They further employ structural equation modeling to map relationships among user characteristics, system-induced effects, and miscalibration, and demonstrate that oversampling underrepresented groups can mitigate stereotypes and improve calibration and quality, albeit with trade-offs. The work provides a principled toolkit for diagnosing and mitigating representation-related harms in recommender systems and highlights the importance of addressing data underrepresentation.

Abstract

Despite the benefits of personalizing items and information tailored to users' needs, it has been found that recommender systems tend to introduce biases that favor popular items or certain categories of items, and dominant user groups. In this study, we aim to characterize the systematic errors of a recommendation system and how they manifest in various accountability issues, such as stereotypes, biases, and miscalibration. We propose a unified framework that distinguishes the sources of prediction errors into a set of key measures that quantify the various types of system-induced effects, both at the individual and collective levels. Based on our measuring framework, we examine the most widely adopted algorithms in the context of movie recommendation. Our research reveals three important findings: (1) Differences between algorithms: recommendations generated by simpler algorithms tend to be more stereotypical but less biased than those generated by more complex algorithms. (2) Disparate impact on groups and individuals: system-induced biases and stereotypes have a disproportionate effect on atypical users and minority groups (e.g., women and older users). (3) Mitigation opportunity: using structural equation modeling, we identify the interactions between user characteristics (typicality and diversity), system-induced effects, and miscalibration. We further investigate the possibility of mitigating system-induced effects by oversampling underrepresented groups and individuals, which was found to be effective in reducing stereotypes and improving recommendation quality. Our research is the first systematic examination of not only system-induced effects and miscalibration but also the stereotyping issue in recommender systems.

Figures (6)

• Figure 1: The overview of the study. We aim to provide a unified framework for the systematically examining miscalibration, system-induced bias and stereotype. (a) Miscalibration is a category-level error in recommender systems, referring to the discrepancy in distributions between the distributions of actual and predicted preferences over item categories. (b) Our framework proposes to decompose miscalibration into two distinct sources of error, bias and variance. (c) It allows us to capture how two system-induced effects, bias and stereotype, can be associated with miscalibration. (d, e) By measuring the group-level or individual-level disparity, we measure whether miscalibration, bias, and stereotype disproportionately impact groups and individuals. (f) The relationship between user characteristics, system-induced effects, and miscalibration can be better understood through association analysis.
• Figure 2: The algorithmic difference of system-induced effects and recommendation quality over five recommendation algorithms. Two performance results in (a) nDCG@20 and (b) miscalibration@20 show that complex models tend to perform better for item-wise predictions (nDCG@20) but worse at category-wise calibration (miscalibration@20). (c) In Bias-Variance ratio, complex models tend to have a higher variance, indicating the capability of better approximating the variance in actual preference. (d) This in turn leads to lower system-induced stereotype.
• Figure 3: Disparate impact of miscalibration and system-induced effects over groups. (a) Miscalibration: For both demographic attributes, gender and age, five recommendation algorithms on average exhibit higher miscalibration for minority groups, women and older users. (b) System-induced effects: Both bias and stereotype tend to impact minority groups to a greater extent. (c) Bias disparity: At the category-level, user preferences in majority-dominant genres (i.e., categories dominantly preferred by majority group) highlighed as red areas tend to be amplified.
• Figure 4: The individual-level impact of miscalibration, bias/variance effect, stereotype, and inflated diversity in WRMF algorithm for (a) gender and (b) age groups. All individuals are ranked by atypicality in order of the most typical (top) to the most atypical (bottom) users and distinguished by gender (younger and men users in orange; older and women users in purple).
• Figure 5: The relationship between systematic effects and miscalibration across five recommendation algorithms.