Modeling Latent Non-Linear Dynamical System over Time Series

TL;DR

LaNoLem tackles the problem of recovering latent nonlinear dynamical systems from time series by introducing a parsimonious latent state representation governed by $\mathbf{s}(t+1)=\mathbf{A}\mathbf{s}(t)+\mathbf{F}\boldsymbol\phi(\mathbf{s}(t),d_\phi)+\mathbf{b}$ and $\mathbf{y}(t)=\mathbf{C}\mathbf{s}(t)+\mathbf{u}$, where $\boldsymbol\phi(\cdot)$ captures polynomial nonlinearities up to order $d_\phi$. It combines an MDL-based criterion for model complexity with an alternating minimization algorithm that alternates between inference (via an extended Kalman filter and sum-product approximations) and learning (via proximal gradient updates enforcing sparsity on $\mathbf{A}$ and $\mathbf{F}$ and closed-form updates for $\mathbf{C},\mathbf{b},\mathbf{u},\boldsymbol\Gamma,\mathbf{R}$). The method achieves competitive dynamics estimation and superior one-step-ahead prediction on chaotic benchmarks, and case studies show latent nonlinear dynamics yield meaningful interpretations in real data (e.g., seasonal Google Trends). Overall, LaNoLem offers a principled approach to extracting latent nonlinear structure with controllable complexity from time series, enabling robust forecasting and interpretable latent dynamics workups. The practical impact lies in improved prediction stability, better handling of noise and missing data, and the ability to reveal interpretable latent mechanisms driving complex temporal behavior.

Abstract

We study the problem of modeling a non-linear dynamical system when given a time series by deriving equations directly from the data. Despite the fact that time series data are given as input, models for dynamics and estimation algorithms that incorporate long-term temporal dependencies are largely absent from existing studies. In this paper, we introduce a latent state to allow time-dependent modeling and formulate this problem as a dynamics estimation problem in latent states. We face multiple technical challenges, including (1) modeling latent non-linear dynamics and (2) solving circular dependencies caused by the presence of latent states. To tackle these challenging problems, we propose a new method, Latent Non-Linear equation modeling (LaNoLem), that can model a latent non-linear dynamical system and a novel alternating minimization algorithm for effectively estimating latent states and model parameters. In addition, we introduce criteria to control model complexity without human intervention. Compared with the state-of-the-art model, LaNoLem achieves competitive performance for estimating dynamics while outperforming other methods in prediction.

Figures (4)

• Figure 1: Overview of LaNoLem: Given a data sequence $\mathbf{X}$, we extract latent states represented by a non-linear dynamical system, where parameters for dynamics, i.e., $\mathbf{A}$ and $\mathbf{F}$, are sparse.
• Figure 2: Accuracy and robustness for dysts dataset: (a) LaNoLem achieves competitive performance for coefficient error. (b) LaNoLem also consistently achieved the lowest prediction error.
• Figure 3: Effectiveness of LaNoLem: LaNoLem can accurately estimate the system and interpolate missing trajectories even when those are noisy and missing data.
• Figure 5: Ablation study: The regularizer in LaNoLem improves (a) the estimation and (b) prediction accuracy of the system.

Theorems & Definitions (6)

• Definition 1
• Definition 2: Moment generating function:$M_X(i)$
• Definition 3: Moment set: $\mathbf{M}\xspace$
• Proposition 1
• proof
• proof