STAR-Rec: Making Peace with Length Variance and Pattern Diversity in Sequential Recommendation

TL;DR

STAR-Rec tackles the challenges of length variance and pattern diversity in sequential recommendation by unifying preference-aware attention with a scalable state-space modeling backbone and routing diverse behavioral patterns through a sequence-level mixture of experts. The method jointly captures static item relationships and temporal dynamics, with an adaptive fusion mechanism that blends PMA and SSM paths and a gating-based MoE for specialized pattern recognition. Comprehensive experiments on four real-world datasets demonstrate state-of-the-art performance, supported by ablation studies showing each component’s contribution and by efficiency analyses confirming practical scalability. The work provides a principled, scalable framework for modeling heterogeneous user behaviors in recommendations, with potential for broader deployment in e-commerce and content platforms.

Abstract

Recent deep sequential recommendation models often struggle to effectively model key characteristics of user behaviors, particularly in handling sequence length variations and capturing diverse interaction patterns. We propose STAR-Rec, a novel architecture that synergistically combines preference-aware attention and state-space modeling through a sequence-level mixture-of-experts framework. STAR-Rec addresses these challenges by: (1) employing preference-aware attention to capture both inherently similar item relationships and diverse preferences, (2) utilizing state-space modeling to efficiently process variable-length sequences with linear complexity, and (3) incorporating a mixture-of-experts component that adaptively routes different behavioral patterns to specialized experts, handling both focused category-specific browsing and diverse category exploration patterns. We theoretically demonstrate how the state space model and attention mechanisms can be naturally unified in recommendation scenarios, where SSM captures temporal dynamics through state compression while attention models both similar and diverse item relationships. Extensive experiments on four real-world datasets demonstrate that STAR-Rec consistently outperforms state-of-the-art sequential recommendation methods, particularly in scenarios involving diverse user behaviors and varying sequence lengths.

Figures (5)

• Figure 1: Overview of STAR-Rec architecture. The model consists of (1) an item embedding layer for initial representations, (2) a unified modeling block that combines preference-aware multi-head attention (PMA) for preference aggregation and state-space model (SSM) for temporal dynamics through an adaptive fusion mechanism, and (3) a mixture-of-experts layer for specialized pattern recognition and prediction.
• Figure 2: Overview of the adaptive fusion mechanism integrating SSM and PMA paths for sequence modeling.
• Figure 3: Illustration of the unified computational pattern between SSM and PMA paths, where both mechanisms can be represented as different attention heads through matrix multiplication operations, with SSM focusing on temporal dependencies and PMA emphasizing preference similarities.
• Figure 4: Ablation study of different components in our model. We evaluate the effectiveness of MoE, PMA, and SSM by removing them, respectively.
• Figure 5: Impact of hidden size $d$ on model performance.