Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SVD-AE: Simple Autoencoders for Collaborative Filtering

Seoyoung Hong, Jeongwhan Choi, Yeon-Chang Lee, Srijan Kumar, Noseong Park

TL;DR

SVD-AE addresses the need for a lightweight, robust collaborative filtering method that does not rely on costly training. It introduces a linear autoencoder whose closed-form solution arises from a truncated SVD of a normalized rating matrix, yielding a low-rank inductive bias that suppresses noise. The approach achieves state-of-the-art or competitive accuracy across several datasets while dramatically reducing computation time, due to avoiding iterative optimization and heavy matrix inversions. This has practical impact for large-scale or latency-sensitive recommender systems, offering reliable performance with minimal training overhead.

Abstract

Collaborative filtering (CF) methods for recommendation systems have been extensively researched, ranging from matrix factorization and autoencoder-based to graph filtering-based methods. Recently, lightweight methods that require almost no training have been recently proposed to reduce overall computation. However, existing methods still have room to improve the trade-offs among accuracy, efficiency, and robustness. In particular, there are no well-designed closed-form studies for \emph{balanced} CF in terms of the aforementioned trade-offs. In this paper, we design SVD-AE, a simple yet effective singular vector decomposition (SVD)-based linear autoencoder, whose closed-form solution can be defined based on SVD for CF. SVD-AE does not require iterative training processes as its closed-form solution can be calculated at once. Furthermore, given the noisy nature of the rating matrix, we explore the robustness against such noisy interactions of existing CF methods and our SVD-AE. As a result, we demonstrate that our simple design choice based on truncated SVD can be used to strengthen the noise robustness of the recommendation while improving efficiency. Code is available at https://github.com/seoyoungh/svd-ae.

SVD-AE: Simple Autoencoders for Collaborative Filtering

TL;DR

SVD-AE addresses the need for a lightweight, robust collaborative filtering method that does not rely on costly training. It introduces a linear autoencoder whose closed-form solution arises from a truncated SVD of a normalized rating matrix, yielding a low-rank inductive bias that suppresses noise. The approach achieves state-of-the-art or competitive accuracy across several datasets while dramatically reducing computation time, due to avoiding iterative optimization and heavy matrix inversions. This has practical impact for large-scale or latency-sensitive recommender systems, offering reliable performance with minimal training overhead.

Abstract

Collaborative filtering (CF) methods for recommendation systems have been extensively researched, ranging from matrix factorization and autoencoder-based to graph filtering-based methods. Recently, lightweight methods that require almost no training have been recently proposed to reduce overall computation. However, existing methods still have room to improve the trade-offs among accuracy, efficiency, and robustness. In particular, there are no well-designed closed-form studies for \emph{balanced} CF in terms of the aforementioned trade-offs. In this paper, we design SVD-AE, a simple yet effective singular vector decomposition (SVD)-based linear autoencoder, whose closed-form solution can be defined based on SVD for CF. SVD-AE does not require iterative training processes as its closed-form solution can be calculated at once. Furthermore, given the noisy nature of the rating matrix, we explore the robustness against such noisy interactions of existing CF methods and our SVD-AE. As a result, we demonstrate that our simple design choice based on truncated SVD can be used to strengthen the noise robustness of the recommendation while improving efficiency. Code is available at https://github.com/seoyoungh/svd-ae.
Paper Structure (39 sections, 3 theorems, 20 equations, 6 figures, 5 tables)

This paper contains 39 sections, 3 theorems, 20 equations, 6 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work & Preliminaries
  3. Lightweight Collaborative Filtering
  4. Generalized Linear Autoencoder for Recommender Systems
  5. Ridge Regression.
  6. Kerenelized Ridge Regression.
  7. SVD-AE
  8. Design Goals.
  9. Method Design.
  10. Low-rank Inductive Bias.
  11. Why SVD-AE Achieves the Best Balance?
  12. The Presence of Noise.
  13. Computational Cost.
  14. Relation to Other Methods
  15. Experiments
  16. ...and 24 more sections

Key Result

Theorem 1

The least squares solutions of the minimum norm of the linear system $\tilde{\mathbf{R}}\mathbf{B} = \mathbf{R}$ is given by where $\tilde{\mathbf{R}}^{+}$ is the the pseudo-inverse of $\tilde{\mathbf{R}}$.

Figures (6)

  • Figure 1: The accuracy, robustness, and computation time of various methods on Gowalla. The $x$-axis indicates each method's accuracy on the original dataset, and the $y$-axis indicates each method's robustness, i.e., its accuracy on the dataset with 5% random interactions added out of all user-item interactions. The computation time is indicated by how fast each method is compared to LightGCN (1x).
  • Figure 2: The performance comparison with different regularization parameters. The $y$-axis of (a) is on a log scale.
  • Figure 3: The smoothing effect of the truncated SVD in reducing noise. All values are normalized between 0 and 1 for better representation. For (a)-(c), 300 users and items from ML-1M are sampled and all interactions of ML-1M are counted for (d) and (e).
  • Figure 4: Performance comparison w.r.t. the only hyperparameter $\gamma$ of SVD-AE. More results in other datasets are in the Appendix \ref{['app:gamma']}.
  • Figure 5: Robustness evaluation against noise level. Solid line for non-AE models, dashed line for AE-based models.
  • ...and 1 more figures

Theorems & Definitions (8)

  • Theorem 1
  • Theorem 2
  • Theorem 3
  • proof
  • proof
  • proof
  • proof
  • proof