Enhancing Predictive Capabilities in Data-Driven Dynamical Modeling with Automatic Differentiation: Koopman and Neural ODE Approaches

TL;DR

A modification of EDMD-DL that concurrently determines both the dictionary of observables and the corresponding approximation of the Koopman operator is presented, which significantly outperforms EDMD-DL and also applies a state-space approach.

Abstract

Data-driven approximations of the Koopman operator are promising for predicting the time evolution of systems characterized by complex dynamics. Among these methods, the approach known as extended dynamic mode decomposition with dictionary learning (EDMD-DL) has garnered significant attention. Here we present a modification of EDMD-DL that concurrently determines both the dictionary of observables and the corresponding approximation of the Koopman operator. This innovation leverages automatic differentiation to facilitate gradient descent computations through the pseudoinverse. We also address the performance of several alternative methodologies. We assess a 'pure' Koopman approach, which involves the direct time-integration of a linear, high-dimensional system governing the dynamics within the space of observables. Additionally, we explore a modified approach where the system alternates between spaces of states and observables at each time step -- this approach no longer satisfies the linearity of the true Koopman operator representation. For further comparisons, we also apply a state space approach (neural ODEs). We consider systems encompassing two and three-dimensional ordinary differential equation systems featuring steady, oscillatory, and chaotic attractors, as well as partial differential equations exhibiting increasingly complex and intricate behaviors. Our framework significantly outperforms EDMD-DL. Furthermore, the state space approach offers superior performance compared to the 'pure' Koopman approach where the entire time evolution occurs in the space of observables. When the temporal evolution of the Koopman approach alternates between states and observables at each time step, however, its predictions become comparable to those of the state space approach.

Figures (10)

• Figure 1: Schematic representation of the state space and function space approaches for the Kuramoto Sivashinsky Equation with a beating traveling wave. The top path evolves the state space $\mathbf{x }$ using a finite-dimensional operator, $\mathbf{F }(t +\delta t)$, while the bottom path updates the observables $\mathbf{\Psi }$ via the Koopman operator, $\mathbf{K }(t +\delta t)$. The vector of observables $\mathbf{\Psi }$ is a function of the full state and a set of dictionary elements from a deep neural network. For this particular case, the PDE has been solved in a grid of 64 points, and 50 trainable dictionary elements complete the set of observables.
• Figure 2: Predictions for the Duffing oscillator with $\lambda=0.5$, $\beta=-1$ and $\alpha=1$. Panel (a) shows the trajectories reconstructed from KDL$_{\textrm{so}}$, KDL$_{\textrm{oo}}$, KDLA$_{\textrm{oo}}$, and the true data. Panel (b) shows the eigenvalues of the Koopman operator for KDLA$_{\textrm{oo}}$. Panel (c) shows the short-time error computed by taking the mean squared difference of 1000 different initial conditions between the exact trajectory and the different models as a function of time. Panel (d) displays the temporal evolution of the same initial condition for different variants KDL$_{\textrm{so}}$ as a function of the number of steps ($m$) that the system has remained within the observable space. To provide a more detailed view, a magnified version of this panel is presented on the right. The dashed lines correspond to the new exact trajectories after reintroducing the state variables through the dictionary. Panel (e) shows the long-time predictions in a $x_1$-$x_2$ plane, in which each initial condition is coloured by its final value of $x_1$ at $t=10$.
• Figure 3: Predictions for the chaotic Rössler attractor with $a=0.1$, $b=0.1$, and $c=9$. Panel (a) shows the temporal evolution of the three components of $\textbf{x}$ forKDL$_{\textrm{oo}}$, KDLA$_{\textrm{oo}}$ and neural ODE. Panels (b-d) show a 3D representation of the Rössler structure, the eigenvalues of the Koopman operator for the KDLA$_{\textrm{oo}}$, and the power spectrum of the model's predictions and true data, respectively.
• Figure 4: Predictions for a three-dimensional model for flow past a cylinder. Panel (a) shows a 3D representation of the trajectory for the true data, KDLA$_{\textrm{oo}}$ and NODE for an initial condition that starts off the attractor, and eventually is attracted onto the limit circle. Panels (b-d) show the eigenvalues of the Koopman operator for the KDLA$_{\textrm{oo}}$, the energy of the system and the power spectrum for the model's predictions and true data, respectively.
• Figure 5: Predictions for the viscous Burger's equation. Panel (a) shows a snapshot of the true (top panel) and predicted dynamics for the KDLA$_{\textrm{oo}}$ and NODE (middle and bottom panels, respectively). Panel (b) compares the value of $u(x,t)$ from the model's predictions and the true state when $u=(x=0,t)$, $u=(x=0.5,t)$, corresponding to the top and bottom panels, respectively. Panels (c)-(d) represent the eigenvalues of the Koopman operator for KDLA$_{\textrm{oo}}$, and the power spectrum of the model's predictions and true data, respectively.