Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Uncertainty in Repeated Implicit Feedback as a Measure of Reliability

Bruno Sguerra, Viet-Anh Tran, Romain Hennequin, Manuel Moussallam

TL;DR

The paper addresses uncertainty in implicit feedback for recommender systems, focusing on music streaming where repeated exposure can reshape user preferences via satiation and the MEE. It introduces a Bayesian beta-binomial framework to quantify both aleatoric and epistemic uncertainty from repeated listening events, deriving per-interaction measures from play counts and recency, namely the posterior mean $\mathbb{E}[L]$ and the 95% HDI. These uncertainty estimates are integrated as reliability weights in an ALS-based recommendation task, demonstrating improvements in Recall@K and NDCG@K on two large music datasets. The work highlights the practical value of incorporating consumption-pattern-driven uncertainty into feedback models and sketches extensions to other domains and model families.

Abstract

Recommender systems rely heavily on user feedback to learn effective user and item representations. Despite their widespread adoption, limited attention has been given to the uncertainty inherent in the feedback used to train these systems. Both implicit and explicit feedback are prone to noise due to the variability in human interactions, with implicit feedback being particularly challenging. In collaborative filtering, the reliability of interaction signals is critical, as these signals determine user and item similarities. Thus, deriving accurate confidence measures from implicit feedback is essential for ensuring the reliability of these signals. A common assumption in academia and industry is that repeated interactions indicate stronger user interest, increasing confidence in preference estimates. However, in domains such as music streaming, repeated consumption can shift user preferences over time due to factors like satiation and exposure. While literature on repeated consumption acknowledges these dynamics, they are often overlooked when deriving confidence scores for implicit feedback. This paper addresses this gap by focusing on music streaming, where repeated interactions are frequent and quantifiable. We analyze how repetition patterns intersect with key factors influencing user interest and develop methods to quantify the associated uncertainty. These uncertainty measures are then integrated as consistency metrics in a recommendation task. Our empirical results show that incorporating uncertainty into user preference models yields more accurate and relevant recommendations. Key contributions include a comprehensive analysis of uncertainty in repeated consumption patterns, the release of a novel dataset, and a Bayesian model for implicit listening feedback.

Uncertainty in Repeated Implicit Feedback as a Measure of Reliability

TL;DR

The paper addresses uncertainty in implicit feedback for recommender systems, focusing on music streaming where repeated exposure can reshape user preferences via satiation and the MEE. It introduces a Bayesian beta-binomial framework to quantify both aleatoric and epistemic uncertainty from repeated listening events, deriving per-interaction measures from play counts and recency, namely the posterior mean and the 95% HDI. These uncertainty estimates are integrated as reliability weights in an ALS-based recommendation task, demonstrating improvements in Recall@K and NDCG@K on two large music datasets. The work highlights the practical value of incorporating consumption-pattern-driven uncertainty into feedback models and sketches extensions to other domains and model families.

Abstract

Recommender systems rely heavily on user feedback to learn effective user and item representations. Despite their widespread adoption, limited attention has been given to the uncertainty inherent in the feedback used to train these systems. Both implicit and explicit feedback are prone to noise due to the variability in human interactions, with implicit feedback being particularly challenging. In collaborative filtering, the reliability of interaction signals is critical, as these signals determine user and item similarities. Thus, deriving accurate confidence measures from implicit feedback is essential for ensuring the reliability of these signals. A common assumption in academia and industry is that repeated interactions indicate stronger user interest, increasing confidence in preference estimates. However, in domains such as music streaming, repeated consumption can shift user preferences over time due to factors like satiation and exposure. While literature on repeated consumption acknowledges these dynamics, they are often overlooked when deriving confidence scores for implicit feedback. This paper addresses this gap by focusing on music streaming, where repeated interactions are frequent and quantifiable. We analyze how repetition patterns intersect with key factors influencing user interest and develop methods to quantify the associated uncertainty. These uncertainty measures are then integrated as consistency metrics in a recommendation task. Our empirical results show that incorporating uncertainty into user preference models yields more accurate and relevant recommendations. Key contributions include a comprehensive analysis of uncertainty in repeated consumption patterns, the release of a novel dataset, and a Bayesian model for implicit listening feedback.
Paper Structure (12 sections, 2 equations, 3 figures, 2 tables)

This paper contains 12 sections, 2 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Uncertainty in Musical implicit feedback
  4. Data
  5. Quantifying Uncertainty in Repeated Consumption
  6. Uncertainty over play counts
  7. Uncertainty over recency
  8. From uncertainty to reliability
  9. Experimental Analysis
  10. Evaluation Datasets
  11. Results and Discussion
  12. Conclusion and Future Work

Figures (3)

  • Figure 1: Left: the evolution of the expected value of the posterior distributions per level of $playcount$. Right: in red the evolution of the 95% HDI of the posteriors per $playcount$ and in blue the number of interactions per $playcount$.
  • Figure 2: Left: the evolution of the expected value of the posteriors obtained for the recency bins together with a shaded 95% HDI. Right: the number of interactions per recency bin.
  • Figure 3: (a) a 3D representation of the average posterior distribution over recency and $playcount$ combinations with a prior $Beta(200,200)$; (b) a 2D projection of the mean posteriors and the corresponding 95% HDI.