Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Core-elements Subsampling for Alternating Least Squares

Dunyao Xue, Mengyu Li, Cheng Meng, Jingyi Zhang

TL;DR

This work tackles the computational bottleneck of alternating least squares (ALS) in missing-value matrix factorization for recommender systems. It introduces Core-ALS, a core-elements subsampling framework that uses sparse sketches to approximate ALS updates while preserving accuracy, supported by per-iteration $(1+\epsilon)$-approximation guarantees and convergence under mild conditions. Theoretical results quantify variance and bias bounds, and a fast variant with partial quicksort accelerates sampling; empirical evidence across synthetic and real datasets (including Netflix) shows substantial speedups with minimal loss in predictive quality. The method demonstrates strong practical impact for large-scale recommender systems by enabling faster training without sacrificing recommendation performance, and suggests future work on memory efficiency and tensor extensions.

Abstract

In this paper, we propose a novel element-wise subset selection method for the alternating least squares (ALS) algorithm, focusing on low-rank matrix factorization involving matrices with missing values, as commonly encountered in recommender systems. While ALS is widely used for providing personalized recommendations based on user-item interaction data, its high computational cost, stemming from repeated regression operations, poses significant challenges for large-scale datasets. To enhance the efficiency of ALS, we propose a core-elements subsampling method that selects a representative subset of data and leverages sparse matrix operations to approximate ALS estimations efficiently. We establish theoretical guarantees for the approximation and convergence of the proposed approach, showing that it achieves similar accuracy with significantly reduced computational time compared to full-data ALS. Extensive simulations and real-world applications demonstrate the effectiveness of our method in various scenarios, emphasizing its potential in large-scale recommendation systems.

Core-elements Subsampling for Alternating Least Squares

TL;DR

This work tackles the computational bottleneck of alternating least squares (ALS) in missing-value matrix factorization for recommender systems. It introduces Core-ALS, a core-elements subsampling framework that uses sparse sketches to approximate ALS updates while preserving accuracy, supported by per-iteration -approximation guarantees and convergence under mild conditions. Theoretical results quantify variance and bias bounds, and a fast variant with partial quicksort accelerates sampling; empirical evidence across synthetic and real datasets (including Netflix) shows substantial speedups with minimal loss in predictive quality. The method demonstrates strong practical impact for large-scale recommender systems by enabling faster training without sacrificing recommendation performance, and suggests future work on memory efficiency and tensor extensions.

Abstract

In this paper, we propose a novel element-wise subset selection method for the alternating least squares (ALS) algorithm, focusing on low-rank matrix factorization involving matrices with missing values, as commonly encountered in recommender systems. While ALS is widely used for providing personalized recommendations based on user-item interaction data, its high computational cost, stemming from repeated regression operations, poses significant challenges for large-scale datasets. To enhance the efficiency of ALS, we propose a core-elements subsampling method that selects a representative subset of data and leverages sparse matrix operations to approximate ALS estimations efficiently. We establish theoretical guarantees for the approximation and convergence of the proposed approach, showing that it achieves similar accuracy with significantly reduced computational time compared to full-data ALS. Extensive simulations and real-world applications demonstrate the effectiveness of our method in various scenarios, emphasizing its potential in large-scale recommendation systems.

Paper Structure

This paper contains 19 sections, 5 theorems, 27 equations, 20 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Alternating Least Squares Algorithm
  4. Core-Elements For Penalized Regression Splines
  5. Methods
  6. Theoretical Results
  7. Approximation Guarantee
  8. Convergence Guarantee
  9. Time Complexity Analysis
  10. Simulation Studies
  11. Estimation Accuracy under Different Parameter Settings
  12. Convergence Performance
  13. Computing Time
  14. Real Data Analysis
  15. Netflix Competition Data
  16. ...and 4 more sections

Key Result

Proposition 1

Let $(t)$ denote the $t$th iteration and $\boldsymbol{L}_{U_i}^{(t)} = \boldsymbol{\widetilde{M}}_{I_i^U}^{(t)} - \boldsymbol{\widetilde{M}}_{I_i^U}^{(t)^{*}}$. Taylor expansions of $\mathbb{E}(\|\widetilde{\boldsymbol{u}}_i^{(t+1)}-\boldsymbol{u}_i^{(t+1)}\|^2)$ at $\boldsymbol{L}_{U_i}^{(t)}$ near where and Here, the terms $\boldsymbol{V}_u^{(t)}$ and $\boldsymbol{B}_u^{(t)}$ represent the upp

Figures (20)

  • Figure 1: Illustration of matrix factorization. Values less than $5\%$ of the maximum value are displayed as white, showing that both $\boldsymbol{U}$ and $\boldsymbol{M}$ are numerically sparse.
  • Figure 2: Schematic of the flow of the Core-ALS.
  • Figure 3: Fast variant of the Core-ALS.
  • Figure 6: ReMSE with respect to the number of iterations.
  • Figure 8: Comparison of bark texture image restoration tasks.
  • ...and 15 more figures

Theorems & Definitions (5)

  • Proposition 1
  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Theorem 4