M2Rec: Multi-scale Mamba for Efficient Sequential Recommendation

TL;DR

M2Rec addresses the efficiency and multi-scale pattern challenges of sequentialRecommendation by integrating an adaptive FFT-based frequency analysis with a Mamba encoder, augmented by LLM-derived semantic embeddings and a learnable gate for multimodal fusion. The approach captures daily to monthly user patterns, denoises inputs, enriches item representations with textual semantics, and dynamically balances temporal, frequency, and semantic signals. Empirical results across diverse datasets demonstrate state-of-the-art HR@10, NDCG@10, and MRR@10, along with reduced inference time compared to Transformer baselines and robustness to noise. The work offers a scalable framework for combining signal processing, language-model semantics, and efficient sequence modeling to advance practical personalized recommendations.

Abstract

Sequential recommendation systems aim to predict users' next preferences based on their interaction histories, but existing approaches face critical limitations in efficiency and multi-scale pattern recognition. While Transformer-based methods struggle with quadratic computational complexity, recent Mamba-based models improve efficiency but fail to capture periodic user behaviors, leverage rich semantic information, or effectively fuse multimodal features. To address these challenges, we propose \model, a novel sequential recommendation framework that integrates multi-scale Mamba with Fourier analysis, Large Language Models (LLMs), and adaptive gating. First, we enhance Mamba with Fast Fourier Transform (FFT) to explicitly model periodic patterns in the frequency domain, separating meaningful trends from noise. Second, we incorporate LLM-based text embeddings to enrich sparse interaction data with semantic context from item descriptions. Finally, we introduce a learnable gate mechanism to dynamically balance temporal (Mamba), frequency (FFT), and semantic (LLM) features, ensuring harmonious multimodal fusion. Extensive experiments demonstrate that \model\ achieves state-of-the-art performance, improving Hit Rate@10 by 3.2\% over existing Mamba-based models while maintaining 20\% faster inference than Transformer baselines. Our results highlight the effectiveness of combining frequency analysis, semantic understanding, and adaptive fusion for sequential recommendation. Code and datasets are available at: https://anonymous.4open.science/r/M2Rec.

Figures (7)

• Figure 1: The diagram illustrates the multi-scale temporal patterns in user interactions from the MovieLens 1M (ML-1M) dataset.
• Figure 2: The diagram illustrates the configuration of the M$^2$Rec. The left part represents the input data, comprising item sequences with item IDs and item contexts. On the right side, the diagram features the AFFM block and the LLMs responsible for generating embeddings that represent the items.
• Figure 3: Performance comparison of baselines on different groups of data
• Figure 4: Ablation study of M$^2$Rec on Texas
• Figure 5: GPU cost comparison of SASRec, BERT4Rec, LRURec, Mamba4Rec and M$^2$Rec on two datasets