Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mamba Hawkes Process

Anningzhe Gao, Shan Dai, Yan Hu

TL;DR

The paper addresses modeling irregular, asynchronous event sequences by integrating the Mamba state-space architecture into Hawkes processes to capture long-range dependencies and dynamic event interactions. It introduces the Mamba Hawkes Process (MHP) with time-variant recurrence and a learnable intensity function, and presents MHP-E, a hybrid Mamba-Transformer encoder, achieving state-of-the-art performance on diverse datasets. The work provides theoretical connections between state-space models and Hawkes processes and demonstrates improved likelihoods and predictive accuracy, offering a flexible, scalable framework for temporal point process modeling. This approach has practical implications for domains with complex temporal dynamics, including finance, social media, healthcare, and beyond.

Abstract

Irregular and asynchronous event sequences are prevalent in many domains, such as social media, finance, and healthcare. Traditional temporal point processes (TPPs), like Hawkes processes, often struggle to model mutual inhibition and nonlinearity effectively. While recent neural network models, including RNNs and Transformers, address some of these issues, they still face challenges with long-term dependencies and computational efficiency. In this paper, we introduce the Mamba Hawkes Process (MHP), which leverages the Mamba state space architecture to capture long-range dependencies and dynamic event interactions. Our results show that MHP outperforms existing models across various datasets. Additionally, we propose the Mamba Hawkes Process Extension (MHP-E), which combines Mamba and Transformer models to enhance predictive capabilities. We present the novel application of the Mamba architecture to Hawkes processes, a flexible and extensible model structure, and a theoretical analysis of the synergy between state space models and Hawkes processes. Experimental results demonstrate the superior performance of both MHP and MHP-E, advancing the field of temporal point process modeling.

Mamba Hawkes Process

TL;DR

The paper addresses modeling irregular, asynchronous event sequences by integrating the Mamba state-space architecture into Hawkes processes to capture long-range dependencies and dynamic event interactions. It introduces the Mamba Hawkes Process (MHP) with time-variant recurrence and a learnable intensity function, and presents MHP-E, a hybrid Mamba-Transformer encoder, achieving state-of-the-art performance on diverse datasets. The work provides theoretical connections between state-space models and Hawkes processes and demonstrates improved likelihoods and predictive accuracy, offering a flexible, scalable framework for temporal point process modeling. This approach has practical implications for domains with complex temporal dynamics, including finance, social media, healthcare, and beyond.

Abstract

Irregular and asynchronous event sequences are prevalent in many domains, such as social media, finance, and healthcare. Traditional temporal point processes (TPPs), like Hawkes processes, often struggle to model mutual inhibition and nonlinearity effectively. While recent neural network models, including RNNs and Transformers, address some of these issues, they still face challenges with long-term dependencies and computational efficiency. In this paper, we introduce the Mamba Hawkes Process (MHP), which leverages the Mamba state space architecture to capture long-range dependencies and dynamic event interactions. Our results show that MHP outperforms existing models across various datasets. Additionally, we propose the Mamba Hawkes Process Extension (MHP-E), which combines Mamba and Transformer models to enhance predictive capabilities. We present the novel application of the Mamba architecture to Hawkes processes, a flexible and extensible model structure, and a theoretical analysis of the synergy between state space models and Hawkes processes. Experimental results demonstrate the superior performance of both MHP and MHP-E, advancing the field of temporal point process modeling.
Paper Structure (28 sections, 1 theorem, 27 equations, 1 figure, 3 tables, 1 algorithm)

This paper contains 28 sections, 1 theorem, 27 equations, 1 figure, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Neural Hawkes Process
  4. State Space Models
  5. Background
  6. Temporal Point Processes
  7. Hawkes Processes.
  8. State Space Models
  9. Methodology
  10. Mamba Hawkes Process
  11. MHP-extension
  12. Experiments
  13. Baselines
  14. Recurrent Marked Temporal Point Process
  15. Neural Hawkes Process
  16. ...and 13 more sections

Key Result

Proposition 4.1

When $N=1, \boldsymbol{A}=-1, \boldsymbol{B}=1$, the Mamba Hawkes Process recurrence takes the form

Figures (1)

  • Figure 1: Comparison of log-likelihood between THP and MHP. Green lines are MHP, red lines are THP. The left figure represent the training process of Mimic-II, the middle figure is for the synthetic dataset and the right is for the stackoverflow dataset. We can see in both figures MHP outperforms THP

Theorems & Definitions (1)

  • Proposition 4.1