Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Decoupled Marked Temporal Point Process using Neural Ordinary Differential Equations

Yujee Song, Donghyun Lee, Rui Meng, Won Hwa Kim

TL;DR

This work tackles the challenge of modeling marked temporal point processes by decoupling the influence of individual events and learning their continuous dynamics with Neural Ordinary Differential Equations. Each past event e_i contributes an independently evolving hidden state h(t;e_i) that generates a per-event influence μ(t;e_i)used to form the ground intensity λ_g^*(t|H_t) and the mark distribution f^*(k|t) through linear, parallelizable aggregations. The Linear Dec-ODE instantiation enables efficient, fixed-step trajectory computation and exact likelihood-based training, achieving competitive or superior results on multiple real-world datasets while offering interpretable visualizations of event-specific dynamics. The framework's parallel training scheme and explicability open avenues for downstream tasks such as survival analysis and out-of-distribution detection in asynchronous event data.

Abstract

A Marked Temporal Point Process (MTPP) is a stochastic process whose realization is a set of event-time data. MTPP is often used to understand complex dynamics of asynchronous temporal events such as money transaction, social media, healthcare, etc. Recent studies have utilized deep neural networks to capture complex temporal dependencies of events and generate embedding that aptly represent the observed events. While most previous studies focus on the inter-event dependencies and their representations, how individual events influence the overall dynamics over time has been under-explored. In this regime, we propose a Decoupled MTPP framework that disentangles characterization of a stochastic process into a set of evolving influences from different events. Our approach employs Neural Ordinary Differential Equations (Neural ODEs) to learn flexible continuous dynamics of these influences while simultaneously addressing multiple inference problems, such as density estimation and survival rate computation. We emphasize the significance of disentangling the influences by comparing our framework with state-of-the-art methods on real-life datasets, and provide analysis on the model behavior for potential applications.

Decoupled Marked Temporal Point Process using Neural Ordinary Differential Equations

TL;DR

This work tackles the challenge of modeling marked temporal point processes by decoupling the influence of individual events and learning their continuous dynamics with Neural Ordinary Differential Equations. Each past event e_i contributes an independently evolving hidden state h(t;e_i) that generates a per-event influence μ(t;e_i)used to form the ground intensity λ_g^*(t|H_t) and the mark distribution f^*(k|t) through linear, parallelizable aggregations. The Linear Dec-ODE instantiation enables efficient, fixed-step trajectory computation and exact likelihood-based training, achieving competitive or superior results on multiple real-world datasets while offering interpretable visualizations of event-specific dynamics. The framework's parallel training scheme and explicability open avenues for downstream tasks such as survival analysis and out-of-distribution detection in asynchronous event data.

Abstract

A Marked Temporal Point Process (MTPP) is a stochastic process whose realization is a set of event-time data. MTPP is often used to understand complex dynamics of asynchronous temporal events such as money transaction, social media, healthcare, etc. Recent studies have utilized deep neural networks to capture complex temporal dependencies of events and generate embedding that aptly represent the observed events. While most previous studies focus on the inter-event dependencies and their representations, how individual events influence the overall dynamics over time has been under-explored. In this regime, we propose a Decoupled MTPP framework that disentangles characterization of a stochastic process into a set of evolving influences from different events. Our approach employs Neural Ordinary Differential Equations (Neural ODEs) to learn flexible continuous dynamics of these influences while simultaneously addressing multiple inference problems, such as density estimation and survival rate computation. We emphasize the significance of disentangling the influences by comparing our framework with state-of-the-art methods on real-life datasets, and provide analysis on the model behavior for potential applications.
Paper Structure (34 sections, 17 equations, 8 figures, 7 tables)

This paper contains 34 sections, 17 equations, 8 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Decoupling Marked Time Point Process
  4. Hidden State Dynamics
  5. Ground intensity function
  6. Conditional Probability for Marks
  7. Linear Dec-ODE
  8. Combining Influences from Past Events
  9. Training Objective
  10. Training Scheme
  11. Related Works
  12. Experiment
  13. Experiment Setup
  14. Comparisons of Results
  15. Explainability of Dec-ODE
  16. ...and 19 more sections

Figures (8)

  • Figure 1: A visualization of the overall framework. Hidden states from each event $e_i$ are independently propagated and decoded into trajectories of $\mu(t;e_i)$ and $\hat{f}_k (k|t, e_i)$. Each trajectory represents the effect of each event on the MTPP, and the MTPP can be reconstructed by combining all trajectories.
  • Figure 2: Visualization of equation \ref{['eqn:multi-integral']}. Different combinations of $\mu(t;e_i)$ can be selected for calculating $\lambda ^*_g(t)$ and $f^*(t)$ conditioned on different $\mathcal{H}_{t_i}$ in parallel.
  • Figure 3: Visualization of propagated $\hat{f}^*(k|t, e_i)$ in StackOverflow experiment. The each axis represent time, event type, and the magnitude. The change of influence through time can be easily observed.
  • Figure 4: Visualization of patterns found in Retweet dataset. (a) $\mu(t;e_i)$ with different marks, (b) Influence of marks on each other, (c) Actual event marks with respect to time.
  • Figure 5: Imputation experiment done using Stackoverflow dataset, where from $10\%$ to $40\%$ of data are randomly dropped.
  • ...and 3 more figures