Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dissertation: On the Theoretical Foundation of Model Comparison and Evaluation for Recommender System

Dong Li

Abstract

Recommender systems have become increasingly important with the rise of the web as a medium for electronic and business transactions. One of the key drivers of this technology is the ease with which users can provide feedback about their likes and dislikes through simple clicks of a mouse. This feedback is commonly collected in the form of ratings, but can also be inferred from a user's browsing and purchasing history. Recommender systems utilize users' historical data to infer customer interests and provide personalized recommendations. The basic principle of recommendations is that significant dependencies exist between user- and item-centric activity, which can be learned in a data-driven manner to make accurate predictions. Collaborative filtering is one family of recommendation algorithms that uses ratings from multiple users to predict missing ratings or uses binary click information to predict potential clicks. However, recommender systems can be more complex and incorporate auxiliary data such as content-based attributes, user interactions, and contextual information.

Dissertation: On the Theoretical Foundation of Model Comparison and Evaluation for Recommender System

Abstract

Recommender systems have become increasingly important with the rise of the web as a medium for electronic and business transactions. One of the key drivers of this technology is the ease with which users can provide feedback about their likes and dislikes through simple clicks of a mouse. This feedback is commonly collected in the form of ratings, but can also be inferred from a user's browsing and purchasing history. Recommender systems utilize users' historical data to infer customer interests and provide personalized recommendations. The basic principle of recommendations is that significant dependencies exist between user- and item-centric activity, which can be learned in a data-driven manner to make accurate predictions. Collaborative filtering is one family of recommendation algorithms that uses ratings from multiple users to predict missing ratings or uses binary click information to predict potential clicks. However, recommender systems can be more complex and incorporate auxiliary data such as content-based attributes, user interactions, and contextual information.

Paper Structure

This paper contains 117 sections, 5 theorems, 183 equations, 15 figures, 20 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Introduction to Recommender System
  3. Goals of Recommender System
  4. Design of Offline Recommender System
  5. Implicit Feedback Data
  6. Training Testing Data Splitting
  7. Evaluation of Offline Ranking-based Recommender System
  8. Collaborative Filtering Model
  9. Organization
  10. Sampling-Based Evaluation
  11. Introduction
  12. Problem Formulation
  13. Leave-one-out Recommendation Evaluation
  14. Top-K Evaluation Metrics
  15. Sampling-based Top-K Metrics Estimation Problem
  16. ...and 102 more sections

Key Result

Theorem 1

Let us assume we have two global Recall curves (empirical cumulative distribution), $T^{(1)}_{Recall@K}$ and $T^{(2)}_{Recall@K}$, and assume one curve dominates the other one, i.e., $T^{(1)}_{Recall@K} \geq T^{(2)}_{Recall@K}$ for any $1\leq K \leq N$; then, for their corresponding sampling curve a

Figures (15)

  • Figure 1: Interaction matrices of explicit and implicit feedback. Figure obtained from dnnrec.
  • Figure 2: Global vs Sampling Top-$K$ Hit-Ratio on $yelp$ dataset. To display the details clearly, we zoom in (a) the global recall curve and (b) the sampling recall curve at different range scales, to $(c)$ and $(d)$ respectively. Comparing two figures, we can easily conclude that sampling evaluation maintains the same curve trend as global evaluation for different algorithms even at a small error range.
  • Figure 3: Curve Relationship of model $EASE$ on $yelp$ dataset. $T_{Recall@K}$ is the top-K global Recall curve; $T^S_{Recall@K}$ is the sampling top-K Recall curve shown in global scale (by baseline \ref{['eq:fk1']}); $\tilde{P}(R)(zoom out 5X)$ is the empirical user ranking distribution, where we make it 5 times larger for displaying purpose.
  • Figure 4: Beta distributions and empirical user rank distribution $\tilde{P}(R)$
  • Figure 5: Relative error w.r.t. $f(k;a=1)$. example on yelp dataset, n= 100.
  • ...and 10 more figures

Theorems & Definitions (11)

  • Theorem 1: Sampling Theorem
  • proof
  • Proposition 1
  • Definition 1
  • Definition 2
  • Theorem 2
  • Theorem 3
  • proof
  • proof
  • Lemma 1: Location of Last Point
  • ...and 1 more