Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Probabilistic Learning of Multivariate Time Series with Temporal Irregularity

Yijun Li, Cheuk Hang Leung, Qi Wu

TL;DR

This work addresses probabilistic forecasting for multivariate time series with temporal irregularity by introducing the Recurrent Flow Network (RFN), a two-layer framework that separates marginal temporal dynamics from a non-parametric, joint distribution learned via a dynamic conditional Continuous Normalizing Flow. RFN supports both Syn-MTS and Asyn-MTS by conditioning a flow-based joint model on latent states derived from irregular observations, enabling sampling at arbitrary continuous times. Across synthetic GBM paths and real datasets (MuJoCo, USHCN, NASDAQ), RFN achieves superior calibration and sharpness compared to Gaussian baselines, demonstrating robustness and practical impact for irregularly sampled multivariate time series.

Abstract

Probabilistic forecasting of multivariate time series is essential for various downstream tasks. Most existing approaches rely on the sequences being uniformly spaced and aligned across all variables. However, real-world multivariate time series often suffer from temporal irregularities, including nonuniform intervals and misaligned variables, which pose significant challenges for accurate forecasting. To address these challenges, we propose an end-to-end framework that models temporal irregularities while capturing the joint distribution of variables at arbitrary continuous-time points. Specifically, we introduce a dynamic conditional continuous normalizing flow to model data distributions in a non-parametric manner, accommodating the complex, non-Gaussian characteristics commonly found in real-world datasets. Then, by leveraging a carefully factorized log-likelihood objective, our approach captures both temporal and cross-sectional dependencies efficiently. Extensive experiments on a range of real-world datasets demonstrate the superiority and adaptability of our method compared to existing approaches.

Probabilistic Learning of Multivariate Time Series with Temporal Irregularity

TL;DR

This work addresses probabilistic forecasting for multivariate time series with temporal irregularity by introducing the Recurrent Flow Network (RFN), a two-layer framework that separates marginal temporal dynamics from a non-parametric, joint distribution learned via a dynamic conditional Continuous Normalizing Flow. RFN supports both Syn-MTS and Asyn-MTS by conditioning a flow-based joint model on latent states derived from irregular observations, enabling sampling at arbitrary continuous times. Across synthetic GBM paths and real datasets (MuJoCo, USHCN, NASDAQ), RFN achieves superior calibration and sharpness compared to Gaussian baselines, demonstrating robustness and practical impact for irregularly sampled multivariate time series.

Abstract

Probabilistic forecasting of multivariate time series is essential for various downstream tasks. Most existing approaches rely on the sequences being uniformly spaced and aligned across all variables. However, real-world multivariate time series often suffer from temporal irregularities, including nonuniform intervals and misaligned variables, which pose significant challenges for accurate forecasting. To address these challenges, we propose an end-to-end framework that models temporal irregularities while capturing the joint distribution of variables at arbitrary continuous-time points. Specifically, we introduce a dynamic conditional continuous normalizing flow to model data distributions in a non-parametric manner, accommodating the complex, non-Gaussian characteristics commonly found in real-world datasets. Then, by leveraging a carefully factorized log-likelihood objective, our approach captures both temporal and cross-sectional dependencies efficiently. Extensive experiments on a range of real-world datasets demonstrate the superiority and adaptability of our method compared to existing approaches.
Paper Structure (41 sections, 2 theorems, 58 equations, 17 figures, 8 tables, 2 algorithms)

This paper contains 41 sections, 2 theorems, 58 equations, 17 figures, 8 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. On Handling Temporal Irregularity
  3. On Modeling Joint Data Distribution
  4. Our Approach and Contributions
  5. Backgrounds
  6. Neural Ordinary Differential Equations
  7. Unconditional Normalizing Flow
  8. Data Structure & Problem Statement
  9. Model Framework
  10. Marginal Learning Layer
  11. Joint Synchronous Learning Layer
  12. Factorizing Log-likelihood Objective
  13. Conditional Flow Representation
  14. Time-varying Specification
  15. Training and Sampling
  16. ...and 26 more sections

Key Result

Lemma 4.1

Let $\tilde{\mathrm{z}}(s)=[\mathrm{z}(s),\mathrm{y}(s)]^{\top}$ be a finite continuous random variable, and the probability density function of $\tilde{\mathrm{z}}(s)$ is $p(\tilde{\mathrm{z}}(s))=p(\mathrm{z}(s),\mathrm{y}(s))$ which depends on flow time $s$, where $s_{0}\leq s\leq s_{1}$. Given t where $s_0 \leq s \leq s_1$, $\mathrm{z}(s_1)=\mathrm{x}$, $\mathrm{y}(s_1)=\mathrm{y}$, and $\math

Figures (17)

  • Figure 1: (a) and (b) are examples of Syn-MTS and Asyn-MTS where observed data points are marked as solid circle dots. While the time intervals between consecutive observation times are unevenly spaced in both cases, component observations of the Syn-MTS sample path are always aligned. In contrast, in the Asyn-MTS case, no observation time has complete observations. This demonstrates that uneven spacing originates at the univariate level, while asynchrony arises exclusively in the multivariate context.
  • Figure 2: The data structure and notations for one instance.
  • Figure 3: The framework of RFNs for (a) Syn-MTS and (b) Asyn-MTS. In both cases, there are two component variables $X^1_t, X^2_t$. The solid points in different colors indicate they are observations of different variables. In the marginal learning layer, the hidden states will be updated only when at least one variable has an observation, e.g., from $\mathrm{h}(t_{1-})$ to $\mathrm{h}(t_{1+})$. In the joint learning layer, the base distribution parameters at each time are $\mu_t$ and $\Sigma_t$ (for the Syn-MTS case) or $\mu_t^d$ and $\Sigma_t^d$ (for the Asyn-MTS case), which is learned from hidden state $\mathrm{h}(t_{1-})$. The conditional CNF transforms the data points following the unknown distribution to the base distribution, for which likelihoods are easy to compute.
  • Figure 4: The framework of conditional CNF.
  • Figure 5: (a) Three representative sample paths. (b) The sample correlation matrix at observed time points 0.3, 0.6, 0.9.
  • ...and 12 more figures

Theorems & Definitions (6)

  • Definition 1
  • Definition 2
  • Lemma 4.1
  • proof
  • Proposition 4.1
  • proof