Deep Generative Modeling for Identification of Noisy, Non-Stationary Dynamical Systems

TL;DR

The method, dynamic SINDy, combines variational inference with SINDy (sparse identification of nonlinear dynamics) to model time-varying coefficients of sparse ODEs, expanding on previous methods for autonomous systems.

Abstract

A significant challenge in many fields of science and engineering is making sense of time-dependent measurement data by recovering governing equations in the form of differential equations. We focus on finding parsimonious ordinary differential equation (ODE) models for nonlinear, noisy, and non-autonomous dynamical systems and propose a machine learning method for data-driven system identification. While many methods tackle noisy and limited data, non-stationarity - where differential equation parameters change over time - has received less attention. Our method, dynamic SINDy, combines variational inference with SINDy (sparse identification of nonlinear dynamics) to model time-varying coefficients of sparse ODEs. This framework allows for uncertainty quantification of ODE coefficients, expanding on previous methods for autonomous systems. These coefficients are then interpreted as latent variables and added to the system to obtain an autonomous dynamical model. We validate our approach using synthetic data, including nonlinear oscillators and the Lorenz system, and apply it to neuronal activity data from C. elegans. Dynamic SINDy uncovers a global nonlinear model, showing it can handle real, noisy, and chaotic datasets. We aim to apply our method to a variety of problems, specifically dynamic systems with complex time-dependent parameters.

Figures (13)

• Figure 1: (A). Synthetic dataset to test dynamic SINDy with non-autonomous harmonic oscillators (Eq. (\ref{['eq_nonlinear_harmonic_osc']})). Top: Example (SINDy) coefficient time series $A(t)$; Bottom: corresponding trajectories in phase space (B). Dynamic SINDy general architecture schematic; two DVAEs shown as example.
• Figure 1: (A) Schematic of timeVAE architecture from timeVAE; (B) Schematic of dynamic HyperSINDy architecture described in SM Sec. 1.2.2
• Figure 2: Dynamic SINDy generates coefficient time series that match ground truth for non-autonomous harmonic oscillators (Eq. (\ref{['eq_nonlinear_harmonic_osc']})). (a)-(f) different examples of time-varying $A(t)$, $B(t)$.
• Figure 2: Time-varying coefficients (above) and corresponding dynamics (below) for the non-autonomous harmonic oscillator of Eq. (3) (main text). Complementary to Figure 1A (main text)
• Figure 3: (A) Dynamic SINDy generates coefficient time series for different levels of Gaussian noise in the coefficient. (B) Inferred noise (standard deviation, or std) scales with ground truth Gaussian noise for different time-varying coefficients. (a) std computed over many generated samples, then averaged (b) std computed over time, then averaged over samples (see Sec. 4.2)