Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The MovieLens Beliefs Dataset: Collecting Pre-Choice Data for Online Recommender Systems

Guy Aridor, Duarte Goncalves, Ruoyan Kong, Daniel Kluver, Joseph Konstan

TL;DR

The paper addresses how recommender systems influence consumer choices by collecting pre-choice beliefs about unexperienced items. It introduces a simple $\text{economic}$ decision-making framework to guide belief elicitation and links beliefs to observed ratings and recommendations, enabling analysis of how recommendations shift beliefs and subsequent consumption. Implemented on the MovieLens platform for over a year, the approach constructs an $M^t$ of about $100y$ items and gathers $B_i^t$ beliefs alongside traditional ratings and recommendation logs, with open access to the dataset. The authors analyze data quality, selection bias, and product-space coverage, demonstrating the dataset’s usefulness for prototyping belief-aware algorithms and for evaluating serendipity and belief-driven consumption in real-world settings. Overall, the work provides a scalable methodology and a valuable resource for advancing belief-centric evaluation and design in recommender systems.

Abstract

An increasingly important aspect of designing recommender systems involves considering how recommendations will influence consumer choices. This paper addresses this issue by introducing a method for collecting user beliefs about un-experienced items - a critical predictor of choice behavior. We implemented this method on the MovieLens platform, resulting in a rich dataset that combines user ratings, beliefs, and observed recommendations. We document challenges to such data collection, including selection bias in response and limited coverage of the product space. This unique resource empowers researchers to delve deeper into user behavior and analyze user choices absent recommendations, measure the effectiveness of recommendations, and prototype algorithms that leverage user belief data, ultimately leading to more impactful recommender systems. The dataset can be found at https://grouplens.org/datasets/movielens/ml_belief_2024/.

The MovieLens Beliefs Dataset: Collecting Pre-Choice Data for Online Recommender Systems

TL;DR

The paper addresses how recommender systems influence consumer choices by collecting pre-choice beliefs about unexperienced items. It introduces a simple decision-making framework to guide belief elicitation and links beliefs to observed ratings and recommendations, enabling analysis of how recommendations shift beliefs and subsequent consumption. Implemented on the MovieLens platform for over a year, the approach constructs an of about items and gathers beliefs alongside traditional ratings and recommendation logs, with open access to the dataset. The authors analyze data quality, selection bias, and product-space coverage, demonstrating the dataset’s usefulness for prototyping belief-aware algorithms and for evaluating serendipity and belief-driven consumption in real-world settings. Overall, the work provides a scalable methodology and a valuable resource for advancing belief-centric evaluation and design in recommender systems.

Abstract

An increasingly important aspect of designing recommender systems involves considering how recommendations will influence consumer choices. This paper addresses this issue by introducing a method for collecting user beliefs about un-experienced items - a critical predictor of choice behavior. We implemented this method on the MovieLens platform, resulting in a rich dataset that combines user ratings, beliefs, and observed recommendations. We document challenges to such data collection, including selection bias in response and limited coverage of the product space. This unique resource empowers researchers to delve deeper into user behavior and analyze user choices absent recommendations, measure the effectiveness of recommendations, and prototype algorithms that leverage user belief data, ultimately leading to more impactful recommender systems. The dataset can be found at https://grouplens.org/datasets/movielens/ml_belief_2024/.
Paper Structure (11 sections, 2 equations, 1 figure)

This paper contains 11 sections, 2 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Procedure
  3. Economic Model of Decision-Making
  4. The Data Collection Challenge
  5. Choosing the Set of Goods, $M^t$
  6. Choosing Elicited Goods, $B_i^t$
  7. Data
  8. Data Collection Details
  9. Datasets
  10. Descriptive Patterns in the Data
  11. Conclusion

Figures (1)

  • Figure 1: Data Collection Interface