Still Competitive: Revisiting Recurrent Models for Irregular Time Series Prediction

TL;DR

This work examines whether simple recurrent architectures can remain competitive for irregular time series by introducing GRUwE, a Gated Recurrent Unit enhanced with exponential basis functions to model time effects and maintain a compact Markov state for continuous-time prediction. GRUwE updates its state through two resets—time-triggered decay and observation-triggered updates—while reusing the same exponential basis for both state evolution and prediction, enabling efficient online deployment. The authors provide theoretical analysis of the exponential basis functions (including Lipschitz continuity and asymptotic behavior) and conduct extensive experiments across four real-world next-observation datasets and multiple next-event datasets, showing competitive or superior performance with significantly reduced computational costs. They also discuss broader impacts, dataset details, and a comprehensive hyperparameter methodology, highlighting GRUwE’s practicality and potential for future enhancements such as uncertainty modeling.

Abstract

Modeling irregularly sampled multivariate time series is a persistent challenge in domains like healthcare and sensor networks. While recent works have explored a variety of complex learning architectures to solve the prediction problems for irregularly sampled time series, it remains unclear what are the true benefits of some of these architectures, and whether clever modifications of simpler and more efficient RNN-based algorithms are still competitive, i.e. they are on par with or even superior to these methods. In this work, we propose and study GRUwE: Gated Recurrent Unit with Exponential basis functions, that builds upon RNN-based architectures for observations made at irregular times. GRUwE supports both regression-based and event-based predictions in continuous time. GRUwE works by maintaining a Markov state representation of the time series that updates with the arrival of irregular observations. The Markov state update relies on two reset mechanisms: (i) observation-triggered reset, and (ii) time-triggered reset of the GRU state using learnable exponential decays, to support the predictions in continuous time. Our empirical evaluations across several real-world benchmarks on next-observation and next-event prediction tasks demonstrate that GRUwE can indeed achieve competitive to superior performance compared to the recent state-of-the-art (SOTA) methods. Thanks to its simplicity, GRUwE offers compelling advantages: it is easy to implement, requires minimal hyper-parameter tuning efforts, and significantly reduces the computational overhead in the online deployment.

Figures (6)

• Figure 1: Pictorial representation of the problem setup.
• Figure 2: GRUwE's overall architecture.
• Figure 3: Computational cost analysis on retrospective data. (\ref{['fig:train_time']}) compares the train times for all models; (\ref{['fig:gpu_memory']}) compares the peak GPU memory usage during training; (\ref{['fig:inference_time']}) compares the inference times on retrospective data. Lower is better for all plots.
• Figure 4: Comparison of Log-Likelihood (LL) for all the TPP models on next event prediction task. A higher value of LL indicates a better fit.
• Figure 5:

Theorems & Definitions (3)

• Theorem 11.1: Lipschitz Continuity of Exponential Decay Function
• proof
• Remark 1: Exponential Basis Functions Induces Growth and Decay