Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Comprehensive Evaluation of Matrix Factorization Models for Collaborative Filtering Recommender Systems

Jesús Bobadilla, Jorge Dueñas-Lerín, Fernando Ortega, Abraham Gutierrez

TL;DR

This paper tested six representative matrix factorization models, using four collaborative filtering datasets, and showed each convenient matrix factorization model attending to their simplicity, the required prediction quality, the necessary recommendation quality, the desired recommendation novelty and diversity, and the need to explain recommendations.

Abstract

Matrix factorization models are the core of current commercial collaborative filtering Recommender Systems. This paper tested six representative matrix factorization models, using four collaborative filtering datasets. Experiments have tested a variety of accuracy and beyond accuracy quality measures, including prediction, recommendation of ordered and unordered lists, novelty, and diversity. Results show each convenient matrix factorization model attending to their simplicity, the required prediction quality, the necessary recommendation quality, the desired recommendation novelty and diversity, the need to explain recommendations, the adequacy of assigning semantic interpretations to hidden factors, the advisability of recommending to groups of users, and the need to obtain reliability values. To ensure the reproducibility of the experiments, an open framework has been used, and the implementation code is provided.

Comprehensive Evaluation of Matrix Factorization Models for Collaborative Filtering Recommender Systems

TL;DR

This paper tested six representative matrix factorization models, using four collaborative filtering datasets, and showed each convenient matrix factorization model attending to their simplicity, the required prediction quality, the necessary recommendation quality, the desired recommendation novelty and diversity, and the need to explain recommendations.

Abstract

Matrix factorization models are the core of current commercial collaborative filtering Recommender Systems. This paper tested six representative matrix factorization models, using four collaborative filtering datasets. Experiments have tested a variety of accuracy and beyond accuracy quality measures, including prediction, recommendation of ordered and unordered lists, novelty, and diversity. Results show each convenient matrix factorization model attending to their simplicity, the required prediction quality, the necessary recommendation quality, the desired recommendation novelty and diversity, the need to explain recommendations, the adequacy of assigning semantic interpretations to hidden factors, the advisability of recommending to groups of users, and the need to obtain reliability values. To ensure the reproducibility of the experiments, an open framework has been used, and the implementation code is provided.

Paper Structure

This paper contains 6 sections, 16 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methods and Experiments
  3. Results
  4. Discussion
  5. Conclusions
  6. Acknowledgments

Figures (5)

  • Figure 1: Precision recommendation quality results; a) MovieLens100K, b) MovieLens 1M, c) FilmTrust, d) MyAnimeList. The higher the values, the better the results.
  • Figure 2: Recall Recommendation quality results obtained in the MovieLens 1M dataset. The results of the other three considered datasets are very similar to this one; to maintain the paper as short as possible, the results of other datasets are not shown.
  • Figure 3: Normalized Discounted Cumulative Gain recommendation quality results; a) MovieLens100K, b) MovieLens 1M, c) FilmTrust, d) MyAnimeList. The higher the values, the better the results.
  • Figure 4: Diversity beyond accuracy results; a) MovieLens100K, b) MovieLens 1M, c) FilmTrust, d) MyAnimeList. The higher the values, the better the results.
  • Figure 5: Novelty beyond accuracy quality results; a) MovieLens100K, b) MovieLens 1M, c) FilmTrust, d) MyAnimeList. The higher the values, the better the results.