Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond Collaborative Filtering: A Relook at Task Formulation in Recommender Systems

Aixin Sun

TL;DR

The paper argues that traditional RecSys task formulations reduce the problem to static matrix completion of $R(u,i)$ over a $|U|\times|I|$ space, neglecting the dynamic decision-making behind user interactions. It advocates for application-specific, timeline-aware task definitions that incorporate contextual inputs $X$, current session items $I_c$, and historical interactions $U\times I$, to better reflect user decision-making. It analyzes mismatches between common offline datasets and real-world contexts, critiques standard evaluation practices, and emphasizes the need for domain-specific datasets and scenario-aligned evaluation to guide baselines. The authors frame recommender systems as inherently dynamic ranking problems that should balance enduring user preferences with momentary context, urging researchers to adopt scenario-driven task formulations, richer data collection, and evaluation methods suitable for emerging settings like conversational and sequential recommendations. Overall, the work seeks to realign RecSys research with practice by clarifying task scopes, datasets, and evaluation to support more meaningful, context-aware recommendations.

Abstract

Recommender Systems (RecSys) have become indispensable in numerous applications, profoundly influencing our everyday experiences. Despite their practical significance, academic research in RecSys often abstracts the formulation of research tasks from real-world contexts, aiming for a clean problem formulation and more generalizable findings. However, it is observed that there is a lack of collective understanding in RecSys academic research. The root of this issue may lie in the simplification of research task definitions, and an overemphasis on modeling the decision outcomes rather than the decision-making process. That is, we often conceptualize RecSys as the task of predicting missing values in a static user-item interaction matrix, rather than predicting a user's decision on the next interaction within a dynamic, changing, and application-specific context. There exists a mismatch between the inputs accessible to a model and the information available to users during their decision-making process, yet the model is tasked to predict users' decisions. While collaborative filtering is effective in learning general preferences from historical records, it is crucial to also consider the dynamic contextual factors in practical settings. Defining research tasks based on application scenarios using domain-specific datasets may lead to more insightful findings. Accordingly, viable solutions and effective evaluations can emerge for different application scenarios.

Beyond Collaborative Filtering: A Relook at Task Formulation in Recommender Systems

TL;DR

The paper argues that traditional RecSys task formulations reduce the problem to static matrix completion of over a space, neglecting the dynamic decision-making behind user interactions. It advocates for application-specific, timeline-aware task definitions that incorporate contextual inputs , current session items , and historical interactions , to better reflect user decision-making. It analyzes mismatches between common offline datasets and real-world contexts, critiques standard evaluation practices, and emphasizes the need for domain-specific datasets and scenario-aligned evaluation to guide baselines. The authors frame recommender systems as inherently dynamic ranking problems that should balance enduring user preferences with momentary context, urging researchers to adopt scenario-driven task formulations, richer data collection, and evaluation methods suitable for emerging settings like conversational and sequential recommendations. Overall, the work seeks to realign RecSys research with practice by clarifying task scopes, datasets, and evaluation to support more meaningful, context-aware recommendations.

Abstract

Recommender Systems (RecSys) have become indispensable in numerous applications, profoundly influencing our everyday experiences. Despite their practical significance, academic research in RecSys often abstracts the formulation of research tasks from real-world contexts, aiming for a clean problem formulation and more generalizable findings. However, it is observed that there is a lack of collective understanding in RecSys academic research. The root of this issue may lie in the simplification of research task definitions, and an overemphasis on modeling the decision outcomes rather than the decision-making process. That is, we often conceptualize RecSys as the task of predicting missing values in a static user-item interaction matrix, rather than predicting a user's decision on the next interaction within a dynamic, changing, and application-specific context. There exists a mismatch between the inputs accessible to a model and the information available to users during their decision-making process, yet the model is tasked to predict users' decisions. While collaborative filtering is effective in learning general preferences from historical records, it is crucial to also consider the dynamic contextual factors in practical settings. Defining research tasks based on application scenarios using domain-specific datasets may lead to more insightful findings. Accordingly, viable solutions and effective evaluations can emerge for different application scenarios.
Paper Structure (6 sections, 2 figures)

This paper contains 6 sections, 2 figures.

Table of Contents

  1. Introduction
  2. The Established RecSys Task Formulation
  3. User, Model, and Item
  4. Recommenders are Task-Specific
  5. The Mismatching Datasets
  6. Conclusion

Figures (2)

  • Figure 1: Illustration of two rounds of recommendations made to a user: (i) user triggers a recommender with her past interaction history $I_u$ and receives the first set of recommendations $R1$; and (ii) the user interacts with item $i_2$ after a decision-making process $d_1$, and receives the second set of recommendations $R2$. The user then interacts with $i_4$ after another decision-making $d_2$. Accordingly, the item collection is updated with the two new interactions. Note that, $R_1$ and $R_2$ are made with different inputs to the model. Best viewed in color.
  • Figure 2: An illustration of train/test instances using leave-one-out data split with interactions by three example users. All interactions are arranged in chronological order following the global timeline. The last interaction of each user is the test instance, represented by a squared octagonal star. Circles are training interactions. The lower half of the figure shows the train/test instances in a typical offline evaluation. The upper half of the figure shows an ideal simulation of a model trained/updated at time $t_m$, for predicting $u_1$'s test instance. Best viewed in color.