Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Variational Autoencoder for Neural Temporal Point Processes with Dynamic Latent Graphs

Sikun Yang, Hongyuan Zha

TL;DR

This work addresses non-stationary dynamics in asynchronous multivariate event sequences by introducing VAETPP, a variational autoencoder for neural temporal point processes with dynamic latent graphs. The time axis is divided into $K$ sub-intervals, with each interval governed by a latent graph ${z}_{(v,u)}^{k}$ capturing dependencies among event types; a GNN-based encoder builds posterior latents, while a forward RNN defines the prior and a GRNN-based decoder propagates influences to predict inter-event times via a log-normal mixture $p( au|m{ heta})$. Training optimizes the ELBO, $ ext{ELBO}(oldsymbol{ heta},oldsymbol{ ho}) = ext{E}_{q_oldsymbol{ ho}(z|S)}[ extstyle extstyle ext{log }p(S|z, heta)] - ext{KL}ig(q_oldsymbol{ ho}(z|S)ig\ ext{ }|| ext{ }p_oldsymbol{ heta}(z|z^{1:k-1},S^{1:k})ig)$, enabling joint learning of the dynamic graph and event-time distributions. Experiments on NYMVC and Stack Exchange datasets show VAETPP achieves superior negative log-likelihood and RMSE compared to Exponential, RMTPP, FullyNN, LogNormMix, and THP, with interpretable time-varying graphs illustrating changing inter-location influences. The method advances dynamic-graph neural TPPs and offers practical gains for forecasting event times and types in non-stationary settings, while future work may explore automatic interval boundary discovery.

Abstract

Continuously-observed event occurrences, often exhibit self- and mutually-exciting effects, which can be well modeled using temporal point processes. Beyond that, these event dynamics may also change over time, with certain periodic trends. We propose a novel variational auto-encoder to capture such a mixture of temporal dynamics. More specifically, the whole time interval of the input sequence is partitioned into a set of sub-intervals. The event dynamics are assumed to be stationary within each sub-interval, but could be changing across those sub-intervals. In particular, we use a sequential latent variable model to learn a dependency graph between the observed dimensions, for each sub-interval. The model predicts the future event times, by using the learned dependency graph to remove the noncontributing influences of past events. By doing so, the proposed model demonstrates its higher accuracy in predicting inter-event times and event types for several real-world event sequences, compared with existing state of the art neural point processes.

A Variational Autoencoder for Neural Temporal Point Processes with Dynamic Latent Graphs

TL;DR

This work addresses non-stationary dynamics in asynchronous multivariate event sequences by introducing VAETPP, a variational autoencoder for neural temporal point processes with dynamic latent graphs. The time axis is divided into sub-intervals, with each interval governed by a latent graph capturing dependencies among event types; a GNN-based encoder builds posterior latents, while a forward RNN defines the prior and a GRNN-based decoder propagates influences to predict inter-event times via a log-normal mixture . Training optimizes the ELBO, , enabling joint learning of the dynamic graph and event-time distributions. Experiments on NYMVC and Stack Exchange datasets show VAETPP achieves superior negative log-likelihood and RMSE compared to Exponential, RMTPP, FullyNN, LogNormMix, and THP, with interpretable time-varying graphs illustrating changing inter-location influences. The method advances dynamic-graph neural TPPs and offers practical gains for forecasting event times and types in non-stationary settings, while future work may explore automatic interval boundary discovery.

Abstract

Continuously-observed event occurrences, often exhibit self- and mutually-exciting effects, which can be well modeled using temporal point processes. Beyond that, these event dynamics may also change over time, with certain periodic trends. We propose a novel variational auto-encoder to capture such a mixture of temporal dynamics. More specifically, the whole time interval of the input sequence is partitioned into a set of sub-intervals. The event dynamics are assumed to be stationary within each sub-interval, but could be changing across those sub-intervals. In particular, we use a sequential latent variable model to learn a dependency graph between the observed dimensions, for each sub-interval. The model predicts the future event times, by using the learned dependency graph to remove the noncontributing influences of past events. By doing so, the proposed model demonstrates its higher accuracy in predicting inter-event times and event types for several real-world event sequences, compared with existing state of the art neural point processes.
Paper Structure (7 sections, 17 equations, 5 figures, 4 tables)

This paper contains 7 sections, 17 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Models
  4. Related Work
  5. Experiments
  6. Conclusion
  7. Acknowledgments

Figures (5)

  • Figure 1: An example illustrates the email interactions between three individuals, Alice (A), Bob (B), Jane (J). During working time, Alice sends emails (black sticks) frequently with Bob, while they interact less frequently, during non-working time. We treat the sequence of emails from one individual to another as an observed dimension, corresponding to a vertex of the dependency graph. The dynamic graph between the four dimensions, aligned with three subintervals, are shown on the top.
  • Figure 2: (a) The sequence consists of event types and timestamps, which are encoded by the history embedding $\{\mathbf{y}_{t_i}^u\}$ with $u$ being the event type. (b) The decoder captures the inter-event time for a future event, using the log-normal mixture model that is parameterized by the history embeddings. The dynamic graph captures the dependencies between those history embeddings.
  • Figure 3: (a) A fully-connected GNN is used to transform event embeddings into relation embeddings between event types at each timestamp. (b) A GRNN is used to transform the past influences of those related event types into the current embedding according to the dynamic graphs.
  • Figure 4: The relation embeddings are fed into a forward RNN and a backward RNN to encode the influences from the past and future relations, respectively.
  • Figure 5: (a) shows the relative locations of the five boroughs of New York city, Manhattan (MH), Bronx (BX), Brooklyn (BK), Queens (QS), Staten Island (SI); (b-f) show the latent dynamic graph between the five boroughs over five time-intervals, estimated by the VAETPP.