User and Recommender Behavior Over Time: Contextualizing Activity, Effectiveness, Diversity, and Fairness in Book Recommendation

TL;DR

The study addresses how real-world recommender data evolves over time and how this temporal evolution affects evaluation and fairness. It employs a longitudinal, time-windowed analysis of the UCSD Book Graph from Goodreads, retraining four implicit-feedback collaborative filters and comparing them across monthly and 2‑year horizons. Key findings include rising female-author representation, persistent popularity bias in interactions, and divergent, time-dependent fairness and diversity patterns in recommendations, with no clear causal impact from Goodreads' 2011 recommender introduction. The work highlights the importance of temporal context for experimental design, fairness assessments, and the interpretation of offline evaluations, urging temporal audits in dataset documentation and evaluation pipelines.

Abstract

Data is an essential resource for studying recommender systems. While there has been significant work on improving and evaluating state-of-the-art models and measuring various properties of recommender system outputs, less attention has been given to the data itself, particularly how data has changed over time. Such documentation and analysis provide guidance and context for designing and evaluating recommender systems, particularly for evaluation designs making use of time (e.g., temporal splitting). In this paper, we present a temporal explanatory analysis of the UCSD Book Graph dataset scraped from Goodreads, a social reading and recommendation platform active since 2006. We measure the book interaction data using a set of activity, diversity, and fairness metrics; we then train a set of collaborative filtering algorithms on rolling training windows to observe how the same measures evolve over time in the recommendations. Additionally, we explore whether the introduction of algorithmic recommendations in 2011 was followed by observable changes in user or recommender system behavior.

Figures (9)

• Figure 1: Unique entity and interaction counts per month.
• Figure 2: Average genre entropy per user.
• Figure 3: Gini of monthly interactions.
• Figure 4: Author gender representation.
• Figure 5: Monthly unique entity counts in recommendations.