How to Diversify any Personalized Recommender?

TL;DR

This work tackles diversity in Top-N recommender outputs by introducing a personalized pre-processing strategy that modestly augments user profiles with carefully selected items from new categories while optionally pruning existing interactions to maintain profile size. It uses a three-input pipeline (Item_categories, User_categories learned via logistic regression, and Initial_predictions from userKNN) to generate Personalized_items_for_addition, with two variants: one-step (only additions) and two-step (additions plus removals) controlled by a parameter lambda. Evaluations on MIND (news) and GoodBook (books) across seven algorithms show that pre-processing yields higher diversity (calibration and exposure) and improved provider fairness (fair-nDCG) with accuracy comparable to or better than baselines, though effects depend on algorithm and lambda. The approach is architecture-agnostic and can be integrated into existing recommender systems, offering a practical path to mitigating filter bubbles and promoting minority-category exposure, while also raising considerations about user autonomy and transparency in profile alterations.

Abstract

In this paper, we introduce a novel approach to improve the diversity of Top-N recommendations while maintaining accuracy. Our approach employs a user-centric pre-processing strategy aimed at exposing users to a wide array of content categories and topics. We personalize this strategy by selectively adding and removing a percentage of interactions from user profiles. This personalization ensures we remain closely aligned with user preferences while gradually introducing distribution shifts. Our pre-processing technique offers flexibility and can seamlessly integrate into any recommender architecture. We run extensive experiments on two publicly available data sets for news and book recommendations to evaluate our approach. We test various standard and neural network-based recommender system algorithms. Our results show that our approach generates diverse recommendations, ensuring users are exposed to a wider range of items. Furthermore, using pre-processed data for training leads to recommender systems achieving performance levels comparable to, and in some cases, better than those trained on original, unmodified data. Additionally, our approach promotes provider fairness by facilitating exposure to minority categories. Our GitHub code is available at: https://github.com/SlokomManel/How-to-Diversify-any-Personalized-Recommender-

Figures (2)

• Figure 1: Our pre-processing approach and its role in a recommendation architecture. $\{i1, i4, i7, i10, i50\}$ are items in user $u$'s past interactions and $\{c1, c2, c3, c4\}$ are categories of these items. In the one-step variant, personalized items $i2$ and $i6$ are added (green). In the two-step variant, items $i4$ and $i10$ are removed (red).
• Figure 2: Fair-nDCG measured for different top-k recommendation lists. The fair-nDCG-score is measured for the recommendation of the minority categories.