Hyper Hawkes Processes: Interpretable Models of Marked Temporal Point Processes

TL;DR

This work presents a new family MTPP models: the hyper Hawkes process (HHP), which aims to be as flexible and performant as neural MTPPs, while retaining interpretable aspects.

Abstract

Foundational marked temporal point process (MTPP) models, such as the Hawkes process, often use inexpressive model families in order to offer interpretable parameterizations of event data. On the other hand, neural MTPPs models forego this interpretability in favor of absolute predictive performance. In this work, we present a new family MTPP models: the hyper Hawkes process (HHP), which aims to be as flexible and performant as neural MTPPs, while retaining interpretable aspects. To achieve this, the HHP extends the classical Hawkes process to increase its expressivity by first expanding the dimension of the process into a latent space, and then introducing a hypernetwork to allow time- and data-dependent dynamics. These extensions define a highly performant MTPP family, achieving state-of-the-art performance across a range of benchmark tasks and metrics. Furthermore, by retaining the linearity of the recurrence, albeit now piecewise and conditionally linear, the HHP also retains much of the structure of the original Hawkes process, which we exploit to create direct probes into how the model creates predictions. HHP models therefore offer both state-of-the-art predictions, while also providing an opportunity to ``open the box'' and inspect how predictions were generated.

Figures (10)

• Figure 1: Schematic of the proposed hyper Hawkes process (HHP). The history of events are input into a hypernetwork which outputs the dynamics of a Hawkes process. These dynamics then dictate how individual particles, each being spawned from a prior event, evolve over time. At a given point in time, these particles are aggregated to produce the model's predicted intensity.
• Figure 2: Visualizations of interpretability results presented in \ref{['sec:exp:interp']}. Bottom left shows a sequence of events where a blue or orange mark is repeated after a predictable time after a green mark occurs. The top left is the model's predicted marked intensities. Middle left showcases the total DF$\lambda$ values per event, with lines colored by the mark that spawned the particle. Right plots show mark-specific DF$\lambda$ trajectories for four particles in the highlighted time range (30, 70).
• Figure 3: The full hyper Hawkes process architecture. We highlight data that is conditioned on with shaded boxes, and the variables that are updated/used in a single iteration, i.e., when the second observation becomes available. The top row represents the history $\mathcal{H}_t$, the second row represents the hypernetwork recurrence, the third row represents the latent Hawkes process, and the bottom row are the intensities. Note we suppress the arrow from $t_1$ into $\mathbf{x}_{2-}$ for visual clarity.
• Figure 4: Visualization of ten example sequences drawn from the data generating process that was analyzed in \ref{['sec:exp:interp']}. Trigger events are overlaid with dots for better readability.
• Figure 5: Visualization of ten example sequences drawn from the data generating process in the second synthetic scenario. Call (blue) and response (orange) events are overlaid with dots for better readability. Note that a response event can only occur after a call event has happened, and vice versa, regardless of how many or few green events occur in the interim.