Meta-Learning for Neural Network-based Temporal Point Processes
Yoshiaki Takimoto, Yusuke Tanaka, Tomoharu Iwata, Maya Okawa, Hideaki Kim, Hiroyuki Toda, Takeshi Kurashima
TL;DR
This work tackles predicting events in continuous time from short observation windows by introducing a meta-learning framework that learns task representations from short sequences via RNNs and conditions a monotonic neural network (MNN) based intensity on these representations. The intensity is decomposed into periodic and aperiodic components to capture daily/weekly cycles and other non-periodic dynamics, with periodicity enforced through an MNN-based cumulative function and a chosen period $\tau$. Training is performed episodically, using urban context information to shape task representations, enabling effective transfer from tasks with long sequences to those with short sequences while avoiding gradient-based inner-loop adaptations. Empirical results on Bikeshare, Taxi, and Crime datasets show superior long-term prediction accuracy and demonstrate the value of incorporating periodicity and urban context for real-world urban-event forecasting.
Abstract
Human activities generate various event sequences such as taxi trip records, bike-sharing pick-ups, crime occurrence, and infectious disease transmission. The point process is widely used in many applications to predict such events related to human activities. However, point processes present two problems in predicting events related to human activities. First, recent high-performance point process models require the input of sufficient numbers of events collected over a long period (i.e., long sequences) for training, which are often unavailable in realistic situations. Second, the long-term predictions required in real-world applications are difficult. To tackle these problems, we propose a novel meta-learning approach for periodicity-aware prediction of future events given short sequences. The proposed method first embeds short sequences into hidden representations (i.e., task representations) via recurrent neural networks for creating predictions from short sequences. It then models the intensity of the point process by monotonic neural networks (MNNs), with the input being the task representations. We transfer the prior knowledge learned from related tasks and can improve event prediction given short sequences of target tasks. We design the MNNs to explicitly take temporal periodic patterns into account, contributing to improved long-term prediction performance. Experiments on multiple real-world datasets demonstrate that the proposed method has higher prediction performance than existing alternatives.