TransFeat-TPP: An Interpretable Deep Covariate Temporal Point Processes

TL;DR

TransFeat-TPP tackles the interpretability gap in covariate-augmented temporal point processes by introducing Fi-SAN, a covariate-importance module that ranks covariates while preserving expressive Transformer-based modeling of event sequences. The model integrates covariates into three parts: time, type, and covariate embeddings, and learns dependencies via self-attention, with Fi-SAN producing an auxiliary representation and covariate importance scores. Time is modeled with a log-normal mixture decoder, and event types are predicted through a joint representation of dependence and Fi-SAN outputs, using auto-tuned multi-task loss. Experiments on synthetic and real data (PM2.5 alerts and London car accidents) show superior predictive performance and consistent, interpretable covariate rankings, highlighting practical value for covariate-driven decision support in dynamic event systems.

Abstract

The classical temporal point process (TPP) constructs an intensity function by taking the occurrence times into account. Nevertheless, occurrence time may not be the only relevant factor, other contextual data, termed covariates, may also impact the event evolution. Incorporating such covariates into the model is beneficial, while distinguishing their relevance to the event dynamics is of great practical significance. In this work, we propose a Transformer-based covariate temporal point process (TransFeat-TPP) model to improve the interpretability of deep covariate-TPPs while maintaining powerful expressiveness. TransFeat-TPP can effectively model complex relationships between events and covariates, and provide enhanced interpretability by discerning the importance of various covariates. Experimental results on synthetic and real datasets demonstrate improved prediction accuracy and consistently interpretable feature importance when compared to existing deep covariate-TPPs.

Figures (11)

• Figure 1: The model structure of TransFeat-TPP. TransFeat-TPP has three modules: the dependence module, the feature importance module and the decoder module. The dependence module extracts the dependencies among events to obtain the representation of the sequence; the feature importance module encodes the covariates and outputs their feature importance; the decoder module uses the learned representations from two previous modules to predict the next event's timestamp and type.
• Figure 2: Hawkes-1
• Figure 3: Hawkes-2
• Figure 4: Hawkes-3
• Figure 5: Hawkes-1