Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Plug-and-Play Bregman ADMM Module for Inferring Event Branches in Temporal Point Processes

Qingmei Wang, Yuxin Wu, Yujie Long, Jing Huang, Fengyuan Ran, Bing Su, Hongteng Xu

TL;DR

This work tackles the challenge of inferring latent event branching in temporal point processes by introducing a plug-and-play Bregman ADMM (BADMM) module that enforces sparse and low-rank structure on event-transition matrices. The BADMM module is designed to integrate with both classic EM-based Hawkes models and transformer-based neural TPPs, unrolling as an attention-like layer for neural models and refining responsibility matrices in EM steps for classic models. By promoting structured, interpretable branches, the approach improves predictive performance while providing clear insights into triggering patterns, such as isolated events and highly influential events. Empirical results on synthetic and real-world datasets demonstrate consistent gains and enhanced interpretability, with code available for reuse and extension.

Abstract

An event sequence generated by a temporal point process is often associated with a hidden and structured event branching process that captures the triggering relations between its historical and current events. In this study, we design a new plug-and-play module based on the Bregman ADMM (BADMM) algorithm, which infers event branches associated with event sequences in the maximum likelihood estimation framework of temporal point processes (TPPs). Specifically, we formulate the inference of event branches as an optimization problem for the event transition matrix under sparse and low-rank constraints, which is embedded in existing TPP models or their learning paradigms. We can implement this optimization problem based on subspace clustering and sparse group-lasso, respectively, and solve it using the Bregman ADMM algorithm, whose unrolling leads to the proposed BADMM module. When learning a classic TPP (e.g., Hawkes process) by the expectation-maximization algorithm, the BADMM module helps derive structured responsibility matrices in the E-step. Similarly, the BADMM module helps derive low-rank and sparse attention maps for the neural TPPs with self-attention layers. The structured responsibility matrices and attention maps, which work as learned event transition matrices, indicate event branches, e.g., inferring isolated events and those key events triggering many subsequent events. Experiments on both synthetic and real-world data show that plugging our BADMM module into existing TPP models and learning paradigms can improve model performance and provide us with interpretable structured event branches. The code is available at \url{https://github.com/qingmeiwangdaily/BADMM_TPP}.

A Plug-and-Play Bregman ADMM Module for Inferring Event Branches in Temporal Point Processes

TL;DR

This work tackles the challenge of inferring latent event branching in temporal point processes by introducing a plug-and-play Bregman ADMM (BADMM) module that enforces sparse and low-rank structure on event-transition matrices. The BADMM module is designed to integrate with both classic EM-based Hawkes models and transformer-based neural TPPs, unrolling as an attention-like layer for neural models and refining responsibility matrices in EM steps for classic models. By promoting structured, interpretable branches, the approach improves predictive performance while providing clear insights into triggering patterns, such as isolated events and highly influential events. Empirical results on synthetic and real-world datasets demonstrate consistent gains and enhanced interpretability, with code available for reuse and extension.

Abstract

An event sequence generated by a temporal point process is often associated with a hidden and structured event branching process that captures the triggering relations between its historical and current events. In this study, we design a new plug-and-play module based on the Bregman ADMM (BADMM) algorithm, which infers event branches associated with event sequences in the maximum likelihood estimation framework of temporal point processes (TPPs). Specifically, we formulate the inference of event branches as an optimization problem for the event transition matrix under sparse and low-rank constraints, which is embedded in existing TPP models or their learning paradigms. We can implement this optimization problem based on subspace clustering and sparse group-lasso, respectively, and solve it using the Bregman ADMM algorithm, whose unrolling leads to the proposed BADMM module. When learning a classic TPP (e.g., Hawkes process) by the expectation-maximization algorithm, the BADMM module helps derive structured responsibility matrices in the E-step. Similarly, the BADMM module helps derive low-rank and sparse attention maps for the neural TPPs with self-attention layers. The structured responsibility matrices and attention maps, which work as learned event transition matrices, indicate event branches, e.g., inferring isolated events and those key events triggering many subsequent events. Experiments on both synthetic and real-world data show that plugging our BADMM module into existing TPP models and learning paradigms can improve model performance and provide us with interpretable structured event branches. The code is available at \url{https://github.com/qingmeiwangdaily/BADMM_TPP}.
Paper Structure (27 sections, 7 equations, 5 figures, 3 tables)

This paper contains 27 sections, 7 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work and Preliminaries
  3. Temporal Point Processes
  4. Inference of Event Branches
  5. Optimization-driven Model Design
  6. Proposed Method
  7. Transition Matrices within TPPs
  8. Hawkes process
  9. Transformer-based neural TPPs
  10. Over-smoothness issue
  11. BADMM Module for Structured Event Branches
  12. Imposing sparse and low-rank structures
  13. Implementation details
  14. Comparisons with Existing Methods
  15. Experiments
  16. ...and 12 more sections

Figures (5)

  • Figure 1: (a) An illustration of event branches and the corresponding transition matrix. (b) Plug-in strategies of BADMM module for classic and neural TPPs.
  • Figure 2: An illustration of the feed-forward step of our Bregman ADMM module.
  • Figure 3: Visualization of inferred transition matrices. The event sequence is from the Conttime dataset.
  • Figure 4: (a, b) Inferred event branches for "12 Angry Men". (c) The branch triggered by the key sentence (corresponding to the blue region in (b)).
  • Figure 5: Robustness test of our BADMM module.