Time Series Continuous Modeling for Imputation and Forecasting with Implicit Neural Representations

TL;DR

This work presents TimeFlow, a unified framework that models time series as continuous functions using conditional INRs modulated by per-sample codes learned via meta-learning. By integrating Fourier-feature based INRs, shift modulation through a hypernetwork, and optimization-based encoding, TimeFlow handles imputation and forecasting for irregular, unaligned, and unseen time series within a single architecture. Empirically, TimeFlow achieves state-of-the-art or competitive performance against both continuous and discrete baselines across imputation and long-horizon forecasting, including challenging scenarios with incomplete look-back windows and new time series. The results demonstrate TimeFlow’s flexibility and robustness, while highlighting limitations such as slower inference and the need for sufficiently large, homogeneous data to fully exploit the shared INR components.

Abstract

We introduce a novel modeling approach for time series imputation and forecasting, tailored to address the challenges often encountered in real-world data, such as irregular samples, missing data, or unaligned measurements from multiple sensors. Our method relies on a continuous-time-dependent model of the series' evolution dynamics. It leverages adaptations of conditional, implicit neural representations for sequential data. A modulation mechanism, driven by a meta-learning algorithm, allows adaptation to unseen samples and extrapolation beyond observed time-windows for long-term predictions. The model provides a highly flexible and unified framework for imputation and forecasting tasks across a wide range of challenging scenarios. It achieves state-of-the-art performance on classical benchmarks and outperforms alternative time-continuous models.

Figures (18)

• Figure 1: Overview of TimeFlow architecture. Forward pass to approximate the time series $x^{(j)}$. $\sigma$ stands for the ReLU activation function.
• Figure 2: Training and inference procedures of TimeFlow for imputation. During training, for each time series $x^{(j)}$, our observations (red dots $\bullet$) are restricted to the sparsely sampled grid, denoted as $\mathcal{T}^{(j)}_{in}$.During inference, our objective is to infer the values over the dense grids $\mathcal{T}^{(j)}$, on the unobserved data points (such as the blue dots $\bullet$ on the figure).
• Figure 3: Electricity dataset. TimeFlow imputation (blue line) and BRITS imputation (gray line) with 10$\%$ of known point (red points) on the eight first days of samples 35 (top) and 25 (bottom).
• Figure 4: Training and inference procedure of TimeFlow for forecasting. During training (top-figure), for each time series $x^{(j)}$, we observe some look-back window/horizon drawing pairs in the trained period. TimeFlow is trained with \ref{['alg:TimeFlow']} to predict all observed timestamps in this drawing pairs while being conditioned by the observed look-back window.Once TimeFlow is optimized, the objective during inference (bottom-figure) is to infer the horizon over new time windows (blue dots $\bullet$) while being conditioned by the newly observed look-back window (red dots $\bullet$).
• Figure 5: Mean MAE forecasting task results over different horizons in the context of generalization to new time series. Comparison of TimeFlow and PatchTST performances on the Electricity, Traffic and SolarH datasets.