Mamba4Rec: Towards Efficient Sequential Recommendation with Selective State Space Models
Chengkai Liu, Jianghao Lin, Jianling Wang, Hanzhou Liu, James Caverlee
TL;DR
This paper tackles the inefficiency of Transformer-based sequential recommendation by introducing Mamba4Rec, which harnesses selective state space models (SSMs) via the Mamba block to achieve linear-time sequence processing. It presents a framework combining an embedding layer, a Mamba-based sequential block, and a prediction layer, with optional stacking and a position-wise feed-forward network to boost performance without sacrificing efficiency. Experiments on MovieLens-1M and two Amazon datasets show Mamba4Rec outperforms RNN-, CNN-, and Transformer-based baselines in both effectiveness and inference efficiency, including lower memory usage and faster convergence. The results establish selective SSMs as a competitive alternative for long-range sequential modeling in recommender systems, with future work aimed at further tailoring SSMs to recommendation tasks.
Abstract
Sequential recommendation aims to estimate the dynamic user preferences and sequential dependencies among historical user behaviors. Although Transformer-based models have proven to be effective for sequential recommendation, they suffer from the inference inefficiency problem stemming from the quadratic computational complexity of attention operators, especially for long behavior sequences. Inspired by the recent success of state space models (SSMs), we propose Mamba4Rec, which is the first work to explore the potential of selective SSMs for efficient sequential recommendation. Built upon the basic Mamba block which is a selective SSM with an efficient hardware-aware parallel algorithm, we design a series of sequential modeling techniques to further promote model performance while maintaining inference efficiency. Through experiments on public datasets, we demonstrate how Mamba4Rec effectively tackles the effectiveness-efficiency dilemma, outperforming both RNN- and attention-based baselines in terms of both effectiveness and efficiency. The code is available at https://github.com/chengkai-liu/Mamba4Rec.