Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Data

TL;DR

This paper tackles irregular time series with missing data by introducing three stable Neural SDE classes: Langevin-type LSDE, Linear Noise SDE (LNSDE), and Geometric SDE (GSDE). It provides theoretical guarantees (existence/uniqueness of strong solutions) and non-asymptotic robustness results, including Wasserstein contraction bounds that decay exponentially with depth when facing distribution shifts. A controlled path is incorporated into the drift to better capture sequential observations, and ablation studies quantify the gains from this design and from nonlinear diffusion. Empirically, the proposed models achieve state-of-the-art performance on interpolation, forecasting, and classification across four benchmarks and 30 datasets with varying missing data, albeit with higher computational cost than some CDE-based approaches. The work advances robust, stable modeling of irregular time series with principled diffusion design and rigorous stability analysis, offering practical benefits for real-world applications.

Abstract

Irregular sampling intervals and missing values in real-world time series data present challenges for conventional methods that assume consistent intervals and complete data. Neural Ordinary Differential Equations (Neural ODEs) offer an alternative approach, utilizing neural networks combined with ODE solvers to learn continuous latent representations through parameterized vector fields. Neural Stochastic Differential Equations (Neural SDEs) extend Neural ODEs by incorporating a diffusion term, although this addition is not trivial, particularly when addressing irregular intervals and missing values. Consequently, careful design of drift and diffusion functions is crucial for maintaining stability and enhancing performance, while incautious choices can result in adverse properties such as the absence of strong solutions, stochastic destabilization, or unstable Euler discretizations, significantly affecting Neural SDEs' performance. In this study, we propose three stable classes of Neural SDEs: Langevin-type SDE, Linear Noise SDE, and Geometric SDE. Then, we rigorously demonstrate their robustness in maintaining excellent performance under distribution shift, while effectively preventing overfitting. To assess the effectiveness of our approach, we conduct extensive experiments on four benchmark datasets for interpolation, forecasting, and classification tasks, and analyze the robustness of our methods with 30 public datasets under different missing rates. Our results demonstrate the efficacy of the proposed method in handling real-world irregular time series data.

Figures (4)

• Figure 1: Comparison of test losses for Neural SDEs with six different diffusion functions on the 'BasicMotions' dataset at a 50% missing rate
• Figure 2: Comparing stability of test loss during model training with the four datasets at a 50% missing rate with the selected methods (Training 100 epochs without early-stopping.)
• Figure 3: Classification result of the 15 univariate datasets with all four settings
• Figure 4: Classification result of the 15 multivariate datasets with all four settings

Theorems & Definitions (13)

• Proposition 3.4
• Theorem 3.5
• Theorem 3.6
• Remark 3.7
• Theorem A.1
• proof : Proof of Proposition \ref{['thm:sde_solutions']}
• Definition B.1
• Theorem B.2
• Proposition B.3
• proof : Proof of Theorem \ref{['thm:stability_lsde']}