Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Continuous-time convolutions model of event sequences

Vladislav Zhuzhel, Vsevolod Grabar, Galina Boeva, Artem Zabolotnyi, Alexander Stepikin, Vladimir Zholobov, Maria Ivanova, Mikhail Orlov, Ivan Kireev, Evgeny Burnaev, Rodrigo Rivera-Castro, Alexey Zaytsev

TL;DR

This paper proposes COTIC, a method based on an efficient convolution neural network designed to handle the non-uniform occurrence of events over time, allowing a model to capture complex dependencies, including, e.g., the self-excitement effect, with little computational expense.

Abstract

Event sequences often emerge in data mining. Modeling these sequences presents two main challenges: methodological and computational. Methodologically, event sequences are non-uniform and sparse, making traditional models unsuitable. Computationally, the vast amount of data and the significant length of each sequence necessitate complex and efficient models. Existing solutions, such as recurrent and transformer neural networks, rely on parametric intensity functions defined at each moment. These functions are either limited in their ability to represent complex event sequences or notably inefficient. We propose COTIC, a method based on an efficient convolution neural network designed to handle the non-uniform occurrence of events over time. Our paper introduces a continuous convolution layer, allowing a model to capture complex dependencies, including, e.g., the self-excitement effect, with little computational expense. COTIC outperforms existing models in predicting the next event time and type, achieving an average rank of 1.5 compared to 3.714 for the nearest competitor. Furthermore, COTIC`s ability to produce effective embeddings demonstrates its potential for various downstream tasks. Our code is open and available at: https://github.com/VladislavZh/COTIC.

Continuous-time convolutions model of event sequences

TL;DR

This paper proposes COTIC, a method based on an efficient convolution neural network designed to handle the non-uniform occurrence of events over time, allowing a model to capture complex dependencies, including, e.g., the self-excitement effect, with little computational expense.

Abstract

Event sequences often emerge in data mining. Modeling these sequences presents two main challenges: methodological and computational. Methodologically, event sequences are non-uniform and sparse, making traditional models unsuitable. Computationally, the vast amount of data and the significant length of each sequence necessitate complex and efficient models. Existing solutions, such as recurrent and transformer neural networks, rely on parametric intensity functions defined at each moment. These functions are either limited in their ability to represent complex event sequences or notably inefficient. We propose COTIC, a method based on an efficient convolution neural network designed to handle the non-uniform occurrence of events over time. Our paper introduces a continuous convolution layer, allowing a model to capture complex dependencies, including, e.g., the self-excitement effect, with little computational expense. COTIC outperforms existing models in predicting the next event time and type, achieving an average rank of 1.5 compared to 3.714 for the nearest competitor. Furthermore, COTIC`s ability to produce effective embeddings demonstrates its potential for various downstream tasks. Our code is open and available at: https://github.com/VladislavZh/COTIC.
Paper Structure (42 sections, 16 equations, 4 figures, 5 tables)

This paper contains 42 sections, 16 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Temporal Point Processes (TPPs)
  4. Deep Learning for TPPs
  5. Deep Learning for Sequential Data
  6. Research Gap
  7. Methods
  8. Preliminaries
  9. Notation
  10. Intensity Function and Likelihood rasmussen2018lecture
  11. Problem statements
  12. Continuous convolution of event sequences
  13. COTIC model architecture
  14. Kernel Parametrization
  15. Return Time and Event Type Prediction
  16. ...and 27 more sections

Figures (4)

  • Figure 1: Scheme of the forward pass for the continuous-time convolutional layer used in the COTIC model. We present output at a single point in time $t_3$.
  • Figure 2: COTIC Architecture: The diagram on the left (Encoder) illustrates the inner layers of the COTIC model, which utilize the states at the timestamps of the original sequence only. The diagram on the right (Final layer) shows the final layer responsible for obtaining the intensity at every point in time.
  • Figure 3: Sensitivity study results. We vary the number of layers, the kernel size, and the activation function (from left to right) to obtain MAE values for the return time prediction on the Retweet dataset.
  • Figure 4: A graph of the intensity versus time for the artificial data. The blue color indicates the true Hawkes intensity. The orange color indicates the intensity restored via the COTIC model.