Dynamical system reconstruction from partial observations using stochastic dynamics

TL;DR

This work tackles reconstructing stochastic dynamical systems from partially observed data by introducing Double Projection Dynamical System Reconstruction (DPDSR), a variational autoencoder framework with dual encoders that infer both latent states $z_t$ and driving noise $\epsilon_t$. Observations $x_t$ are mapped to a state trajectory $\hat{z}_{1:T}$ and a noise trajectory $\epsilon_{1:T}$, after which the system is evolved with a generative model $z_t = \tanh\left( f(z_{t-1}) + B \epsilon_t \right)$ and $x_t = g(z_t) + \Sigma_\eta \eta_t$. Training optimizes a ELBO-like loss combining reconstruction of observations and latent states with a KL term, while employing teacher forcing every $\tau$ steps to stabilize long-horizon dynamics; regularization supports desirable state scaling. The method is benchmarked on six datasets, including deterministic chaos and real neural/ECG data, and analyzed to reveal two regimes governed by $\tau$: a deterministic-chaos regime at small $\tau$ and a noise-driven regime at larger $\tau$. DPDSR generally outperforms deterministic alternatives on stochastic problems and remains competitive on chaotic systems, offering robust reconstruction under partial observability and insight into the learned attractors. This approach advances practical learning of noise-driven dynamical systems with interpretable long-term behavior, with potential impact on neuroscience and other fields where partial observations hinder traditional modeling.

Abstract

Learning stochastic models of dynamical systems underlying observed data is of interest in many scientific fields. Here we propose a novel method for this task, based on the framework of variational autoencoders for dynamical systems. The method estimates from the data both the system state trajectories and noise time series. This approach allows to perform multi-step system evolution and supports a teacher forcing strategy, alleviating limitations of autoencoder-based approaches for stochastic systems. We demonstrate the performance of the proposed approach on six test problems, covering simulated and experimental data. We further show the effects of the teacher forcing interval on the nature of the internal dynamics, and compare it to the deterministic models with equivalent architecture.

Figures (11)

• Figure 1: Graphical summary of the DPDSR method. (A) Generation, visualized for teacher forcing interval $\tau = 2$. (B) Encoding. For brevity, we use a shorthand for the posterior distributions $q(\epsilon_t)$ instead of $q(\epsilon_t \mid \boldsymbol{x}, \hat{\boldsymbol{z}}, \boldsymbol{\epsilon}_{1:t-1})$.
• Figure 2: Results for the double well dataset. (A) Trajectory in the original state space of the model. The dots represent the stable fixed points of the model. (B) Simulated trajectory in the state space of the DPDSR model (hand-picked dimensions). (C) Original time series, and time series generated by the trained models. Here, and in all other figures, the time is represented in sample indices and not the original model time.
• Figure 3: Main results showing the score (lower = better) for all datasets and methods. Each circle represents one of four initialization of the training method, and the bar shows the mean. The measures from which the score is computed are shown on Fig. \ref{['fig:main-results-ext']}. For visualization purposes, the values are clipped to the upper limit of the shown range.
• Figure 4: Results for the ECG dataset. (A) Example time series: original, and simulated by the proposed stochastic model (DPDSR) and its deterministic variant (SPDSR). (B) Distribution of the interspike intervals (ISI) in the data (gray) and generated by the DPDSR model. The two distributions mostly overlap, but note the non zero bin near interval 300 indicating skipped beat in the simulated data. (C) Distribution of the ISI in the data (gray) and generated by the deterministic SPDSR model. The model generated data show periodic behavior with all ISIs concentrated on interval 168. (D) Distance between the data and model-generated ISI distributions for all methods. Each circle represent a different training initialization, bar shows their mean. The gray circle represent the model used in panels A-C. The match of ISI distributions in the DPDSR model is strongly dependent on the initialization, which affects the proportion of skipped beats.
• Figure 5: Analysis of the attractors of the trained models. (A) Maximal Lyapunov exponent $\lambda_{\text{max}}$ for the models trained on the six datasets. Each point correspond to one attractor. Dashed circle outline represents a chaotic attractor ($\lambda_{\text{max}} > 0$), colored solid outline represents a limit cycle, and black solid outline a fixed point. Size of the circle corresponds to the size of the basin of attraction. (B) Influence of the teacher forcing interval $\tau$ on the attractors in the DPDSR models. Circles show the maximal Lyapunov exponent as in (A). Solid red line shows the score (lower = better). Dashed green line shows the KL divergence of the estimated posterior distribution of the noise to the prior distribution $\mathrm{KL}_\epsilon = D_\mathrm{KL}(q(\boldsymbol{\epsilon} \mid \boldsymbol{x}) \;\|\; p(\boldsymbol{\epsilon}))$. The yellow band indicates the optimal $\tau$ value for the dataset. (C) As in (B), but for the deterministic SPDSR models.