Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HMamba: Hyperbolic Mamba for Sequential Recommendation

Qianru Zhang, Honggang Wen, Wei Yuan, Crystal Chen, Menglin Yang, Siu-Ming Yiu, Hongzhi Yin

TL;DR

Hyperbolic Mamba (HMamba) addresses the dual challenge in sequential recommendation of capturing hierarchical structure while maintaining scalable, linear-time inference. By unifying Lorentz hyperbolic embeddings with selective state-space models, HMamba delivers two variants, HMamba-Full and HMamba-Half, that preserve geometry via stabilized Riemannian operations and curvature-aware discretization. The approach yields consistent accuracy gains (3–11%) over strong baselines and substantially improves efficiency relative to Transformer-based models, enabling real-world deployment. Its principled combination of hyperbolic geometry and linear-time SSM represents a new paradigm for hierarchy-aware sequential modeling with broad applicability to data exhibiting taxonomic and temporal structure.

Abstract

Sequential recommendation systems have become a cornerstone of personalized services, adept at modeling the temporal evolution of user preferences by capturing dynamic interaction sequences. Existing approaches predominantly rely on traditional models, including RNNs and Transformers. Despite their success in local pattern recognition, Transformer-based methods suffer from quadratic computational complexity and a tendency toward superficial attention patterns, limiting their ability to infer enduring preference hierarchies in sequential recommendation data. Recent advances in Mamba-based sequential models introduce linear-time efficiency but remain constrained by Euclidean geometry, failing to leverage the intrinsic hyperbolic structure of recommendation data. To bridge this gap, we propose Hyperbolic Mamba, a novel architecture that unifies the efficiency of Mamba's selective state space mechanism with hyperbolic geometry's hierarchical representational power. Our framework introduces (1) a hyperbolic selective state space that maintains curvature-aware sequence modeling and (2) stabilized Riemannian operations to enable scalable training. Experiments across four benchmarks demonstrate that Hyperbolic Mamba achieves 3-11% improvement while retaining Mamba's linear-time efficiency, enabling real-world deployment. This work establishes a new paradigm for efficient, hierarchy-aware sequential modeling.

HMamba: Hyperbolic Mamba for Sequential Recommendation

TL;DR

Hyperbolic Mamba (HMamba) addresses the dual challenge in sequential recommendation of capturing hierarchical structure while maintaining scalable, linear-time inference. By unifying Lorentz hyperbolic embeddings with selective state-space models, HMamba delivers two variants, HMamba-Full and HMamba-Half, that preserve geometry via stabilized Riemannian operations and curvature-aware discretization. The approach yields consistent accuracy gains (3–11%) over strong baselines and substantially improves efficiency relative to Transformer-based models, enabling real-world deployment. Its principled combination of hyperbolic geometry and linear-time SSM represents a new paradigm for hierarchy-aware sequential modeling with broad applicability to data exhibiting taxonomic and temporal structure.

Abstract

Sequential recommendation systems have become a cornerstone of personalized services, adept at modeling the temporal evolution of user preferences by capturing dynamic interaction sequences. Existing approaches predominantly rely on traditional models, including RNNs and Transformers. Despite their success in local pattern recognition, Transformer-based methods suffer from quadratic computational complexity and a tendency toward superficial attention patterns, limiting their ability to infer enduring preference hierarchies in sequential recommendation data. Recent advances in Mamba-based sequential models introduce linear-time efficiency but remain constrained by Euclidean geometry, failing to leverage the intrinsic hyperbolic structure of recommendation data. To bridge this gap, we propose Hyperbolic Mamba, a novel architecture that unifies the efficiency of Mamba's selective state space mechanism with hyperbolic geometry's hierarchical representational power. Our framework introduces (1) a hyperbolic selective state space that maintains curvature-aware sequence modeling and (2) stabilized Riemannian operations to enable scalable training. Experiments across four benchmarks demonstrate that Hyperbolic Mamba achieves 3-11% improvement while retaining Mamba's linear-time efficiency, enabling real-world deployment. This work establishes a new paradigm for efficient, hierarchy-aware sequential modeling.
Paper Structure (24 sections, 5 theorems, 23 equations, 7 figures, 6 tables, 1 algorithm)

This paper contains 24 sections, 5 theorems, 23 equations, 7 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Method
  3. Preliminary and Problem Definition
  4. Overview of HMamba-Full and HMamba-Half
  5. Learning Embeddings in Hyperbolic Space
  6. Hyperbolic Mamba Encoder Layer
  7. Model Optimization via Hyperbolic Loss
  8. Model Complexity
  9. Theoretical Guarantees of HMamba
  10. Experiments
  11. Experiment Settings
  12. Datasets.
  13. Baselines
  14. Evaluation Metrics
  15. Parameter Settings and Environment Setup
  16. ...and 9 more sections

Key Result

Theorem 2.1

For any sequence length $L > 0$ and embedding dimension $d \geq 1$, the hyperbolic Mamba encoder satisfies: where $C_1, C_2 > 0$ are constants depending on the curvature $k = 1/c$, and $\|\cdot\|_\mathcal{H}$ denotes the hyperbolic norm.

Figures (7)

  • Figure 1: Hyperbolic and Euclidean embeddings of ML-1M hierarchy structure, showing (1) Genre (4 kinds: Action, Comedy, Drama, Sci-Fi, Romance) clusters (central colored sectors) (2) Movies (intermediate ring with genre associations) and (3) Users (peripheral points clustered by preference patterns).
  • Figure 2: The diagram illustrates the configuration of the HMamba. The upper part represents the input data, comprising item sequences with item IDs. On the bottom, the diagram features the Hyperbolic Mamba block and the two types of HMamba frameworks, namely HMamba-Full and HMamba-Half.
  • Figure 3: Performance comparison of baselines on different groups of data
  • Figure 4: Ablation Study on Data with the Sequence Length$\leq$ 200
  • Figure 5: GPU cost comparison of SASRec, BERT4Rec, LRURec, Mamba4Rec, HMamba-Full and HMamba-Half on each epoch
  • ...and 2 more figures

Theorems & Definitions (5)

  • Theorem 2.1: Approximation Error Bound
  • Lemma 2.2: Curvature Stability
  • Theorem 2.3: Sample Complexity
  • Theorem 2.4: Parameter Efficiency of Hyperbolic Embeddings
  • Corollary 2.5: Convergence Rate