Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ADiff4TPP: Asynchronous Diffusion Models for Temporal Point Processes

Amartya Mukherjee, Ruizhi Deng, He Zhao, Yuzhen Mao, Leonid Sigal, Frederick Tung

TL;DR

ADiff4TPP tackles temporal point process forecasting by learning latent event representations with a $\beta$-VAE and modeling their joint distribution through an asynchronous diffusion process driven by a matrix-valued schedule $A(s)$. Training uses Conditional Flow Matching with a Diffusion Transformer, solving an ODE $\dot{\mathbf{x}}_s=A'(s)\mathbf{v}_\theta(\mathbf{x}_s,A(s))$ to generate or forecast events, while allowing variable observation/prediction windows. The method achieves state-of-the-art results for next-event prediction and long-horizon forecasting across benchmarks, and its asynchronous diffusion enables faster conditioning on distant future events and efficient inference. By operating in latent space and employing an ODE-based generation, ADiff4TPP provides a scalable, flexible framework for heterogeneous event data with significant practical impact on real-time forecasting tasks.

Abstract

This work introduces a novel approach to modeling temporal point processes using diffusion models with an asynchronous noise schedule. At each step of the diffusion process, the noise schedule injects noise of varying scales into different parts of the data. With a careful design of the noise schedules, earlier events are generated faster than later ones, thus providing stronger conditioning for forecasting the more distant future. We derive an objective to effectively train these models for a general family of noise schedules based on conditional flow matching. Our method models the joint distribution of the latent representations of events in a sequence and achieves state-of-the-art results in predicting both the next inter-event time and event type on benchmark datasets. Additionally, it flexibly accommodates varying lengths of observation and prediction windows in different forecasting settings by adjusting the starting and ending points of the generation process. Finally, our method shows superior performance in long-horizon prediction tasks, outperforming existing baseline methods.

ADiff4TPP: Asynchronous Diffusion Models for Temporal Point Processes

TL;DR

ADiff4TPP tackles temporal point process forecasting by learning latent event representations with a -VAE and modeling their joint distribution through an asynchronous diffusion process driven by a matrix-valued schedule . Training uses Conditional Flow Matching with a Diffusion Transformer, solving an ODE to generate or forecast events, while allowing variable observation/prediction windows. The method achieves state-of-the-art results for next-event prediction and long-horizon forecasting across benchmarks, and its asynchronous diffusion enables faster conditioning on distant future events and efficient inference. By operating in latent space and employing an ODE-based generation, ADiff4TPP provides a scalable, flexible framework for heterogeneous event data with significant practical impact on real-time forecasting tasks.

Abstract

This work introduces a novel approach to modeling temporal point processes using diffusion models with an asynchronous noise schedule. At each step of the diffusion process, the noise schedule injects noise of varying scales into different parts of the data. With a careful design of the noise schedules, earlier events are generated faster than later ones, thus providing stronger conditioning for forecasting the more distant future. We derive an objective to effectively train these models for a general family of noise schedules based on conditional flow matching. Our method models the joint distribution of the latent representations of events in a sequence and achieves state-of-the-art results in predicting both the next inter-event time and event type on benchmark datasets. Additionally, it flexibly accommodates varying lengths of observation and prediction windows in different forecasting settings by adjusting the starting and ending points of the generation process. Finally, our method shows superior performance in long-horizon prediction tasks, outperforming existing baseline methods.
Paper Structure (35 sections, 3 theorems, 31 equations, 5 figures, 8 tables, 2 algorithms)

This paper contains 35 sections, 3 theorems, 31 equations, 5 figures, 8 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Temporal point processes.
  4. Flow Matching
  5. Asynchronous Diffusion for Event Sequence
  6. Event Latent Space Representations
  7. Asynchronous Diffusion Models
  8. Diffusion Transformer (DiT)
  9. Future Event Prediction as Conditional Generation
  10. Noise Schedules and Training Objectives
  11. Conditions on the Noise Schedule
  12. Invertibility and Differentiability of the Noise Schedule
  13. Equivalence of Objective Functions
  14. Experiments
  15. Next Event Prediction
  16. ...and 20 more sections

Key Result

Lemma 1

The flow $\psi(\mathbf{x}_s|\boldsymbol \epsilon )$ in Equation eq:sampling_s_1 governed by a matrix-valued coefficient $A(s)\in (H^1[0,1])^{n\times n}$ remains valid as long as $A(s)$ satisfies condition (4) from Assumption as:a_s.

Figures (5)

  • Figure 1: An overview of diffusion models with asynchronous noise schedules applied to TPP sequences in latent space. We use the size and blurriness of blob to indicate the scale of noise in each event where larger and more blurry blobs represent more noisy data. In the top part of the figure, the latent representations of later events are replaced by Gaussian noise faster than earlier events in the diffusion process (bottom to top). In the generation process (top to bottom), early events will be denoised first before the generation of the future ones. An encoder encodes the duration and category of each single event into a latent representation and a decoder decodes the generated latent representation into the predicted distributions of event duration and categories as shown in the bottom part of the figure.
  • Figure 2: This figure visualizes the synchronous diffusion process when being applied to model TPPs. At each intermediate step in the diffusion or generation process, the model assumes the same scale of noise for each event.
  • Figure 3: Asynchronous noise schedule for an event sequence of length 6. The noise schedule shows that Event 6 (the latest event in the sequence) is the first event to be completely diffused (at flow time $s=\frac{6}{11}$). Event 1 (the earliest event in the sequence) is the last event to start diffusing (at $s=\frac{5}{11}$) and be completely diffused (at $s=1$). Thus, in reverse-flow time (generation), we see that Event 1 is the first event to be completely restored, and Event 6 is the last to be completely restored.
  • Figure 4: Plots of long horizon prediction conducted on three datasets. ADiff4TPP outperforms the baseline methods in each dataset with respect to the OTD metric. The y-axis is scaled to show the difference between baseline methods.
  • Figure 5: Different noise schedules

Theorems & Definitions (7)

  • Lemma 1: Informal
  • Remark 1
  • Remark 2
  • Remark 3
  • Definition 1
  • Lemma 2: Validity of $\psi_s^{-1}$
  • Proposition 1