Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Behavior-Dependent Linear Recurrent Units for Efficient Sequential Recommendation

Chengkai Liu, Jianghao Lin, Hanzhou Liu, Jianling Wang, James Caverlee

TL;DR

RecBLR is proposed, an Efficient Sequential Recommendation Model based on Behavior-Dependent Linear Recurrent Units to accomplish the impossible triangle of the three principles of sequential recommendation simultaneously, exhibiting excellent scalability to datasets with long user interaction histories.

Abstract

Sequential recommender systems aims to predict the users' next interaction through user behavior modeling with various operators like RNNs and attentions. However, existing models generally fail to achieve the three golden principles for sequential recommendation simultaneously, i.e., training efficiency, low-cost inference, and strong performance. To this end, we propose RecBLR, an Efficient Sequential Recommendation Model based on Behavior-Dependent Linear Recurrent Units to accomplish the impossible triangle of the three principles. By incorporating gating mechanisms and behavior-dependent designs into linear recurrent units, our model significantly enhances user behavior modeling and recommendation performance. Furthermore, we unlock the parallelizable training as well as inference efficiency for our model by designing a hardware-aware scanning acceleration algorithm with a customized CUDA kernel. Extensive experiments on real-world datasets with varying lengths of user behavior sequences demonstrate RecBLR's remarkable effectiveness in simultaneously achieving all three golden principles - strong recommendation performance, training efficiency, and low-cost inference, while exhibiting excellent scalability to datasets with long user interaction histories.

Behavior-Dependent Linear Recurrent Units for Efficient Sequential Recommendation

TL;DR

RecBLR is proposed, an Efficient Sequential Recommendation Model based on Behavior-Dependent Linear Recurrent Units to accomplish the impossible triangle of the three principles of sequential recommendation simultaneously, exhibiting excellent scalability to datasets with long user interaction histories.

Abstract

Sequential recommender systems aims to predict the users' next interaction through user behavior modeling with various operators like RNNs and attentions. However, existing models generally fail to achieve the three golden principles for sequential recommendation simultaneously, i.e., training efficiency, low-cost inference, and strong performance. To this end, we propose RecBLR, an Efficient Sequential Recommendation Model based on Behavior-Dependent Linear Recurrent Units to accomplish the impossible triangle of the three principles. By incorporating gating mechanisms and behavior-dependent designs into linear recurrent units, our model significantly enhances user behavior modeling and recommendation performance. Furthermore, we unlock the parallelizable training as well as inference efficiency for our model by designing a hardware-aware scanning acceleration algorithm with a customized CUDA kernel. Extensive experiments on real-world datasets with varying lengths of user behavior sequences demonstrate RecBLR's remarkable effectiveness in simultaneously achieving all three golden principles - strong recommendation performance, training efficiency, and low-cost inference, while exhibiting excellent scalability to datasets with long user interaction histories.
Paper Structure (16 sections, 15 equations, 6 figures, 5 tables, 1 algorithm)

This paper contains 16 sections, 15 equations, 6 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Sequential Recommendation Models
  4. Linear Recurrent Units
  5. Methodology
  6. Overview
  7. Behavior-Dependent Linear Recurrent Units
  8. Hardware-Aware Parallel Scan
  9. Experiments
  10. Experimental Setup
  11. Overall Performance
  12. Efficiency and Scalability Study
  13. Ablation Study
  14. Hyperparameter Sensitivity
  15. Related Work
  16. ...and 1 more sections

Figures (6)

  • Figure 1: The triangle of three gold principles for the model design in sequential recommendation.
  • Figure 2: The overall architecture of RecBLR.
  • Figure 3: Parallel scan for computing hidden representations.
  • Figure 4: GPU memory, training time, and inference time per epoch on XLong with different sequence lengths.
  • Figure 5: Influence of the expansion factor on various metrics.
  • ...and 1 more figures