Tracking and forecasting oscillatory data streams using Koopman autoencoders and Kalman filtering

TL;DR

This work addresses tracking and forecasting time-varying nonlinear dynamical systems from streaming data by fusing Koopman autoencoders with ensemble Kalman filtering (KAE EnKF). The authors develop Adapted Koopsman Autoencoders with latent linear dynamics, a global frequency optimization scheme, and an EnKF in the latent space to efficiently ingest new data and update both states and Koopman modes, enabling robust short-term forecasts and rapid adaptation to nonstationary changes. Through synthetic experiments and a pendulum video, they demonstrate improved tracking, forecasting accuracy, and uncertainty quantification versus full-state filtering and various nonlinear DMD variants, with strong gains in high-dimensional settings due to reduced state dimensionality in the EnKF. The approach offers a scalable, real-time data-driven modeling framework applicable to complex systems such as weather prediction and other geophysical contexts, while also handling multiple frequencies and nonstationary dynamics via latent-space assimilations and frequency-aware training.

Abstract

Data-driven modelling techniques provide a method for deriving models of dynamical systems directly from complicated data streams. However, tracking and forecasting such data streams poses a significant challenge to most methods, as they assume the underlying process and model does not change over time. In this paper, we apply one such data-driven method, the Koopman autoencoder (KAE), to high-dimensional oscillatory data to generate a low-dimensional latent space and model, where the system's dynamics appear linear. This allows one to accurately track and forecast systems where the underlying model may change over time. States and the model in the reduced order latent space can then be efficiently updated as new data becomes available, using data assimilation techniques such as the ensemble Kalman filter (EnKF), in a technique we call the KAE EnKF. We demonstrate that this approach is able to effectively track and forecast time-varying, nonlinear dynamical systems in synthetic examples. We then apply the KAE EnKF to a video of a physical pendulum, and achieve a significant improvement over current state-of-the-art methods. By generating effective latent space reconstructions, we find that we are able to construct accurate short-term forecasts and efficient adaptations to externally forced changes to the pendulum's frequency.

Figures (23)

• Figure 1: States $\mathbf{x}_k$ are encoded via neural network $\mathbf{h}_\phi()$ into latent states $\Tilde{\mathbf{x}}_k$, before being decoded back into the original state by the network $\mathbf{\Tilde{h}}_{\Tilde{\phi}}()$. By constraining the latent space in some way, then optimizing the trainable parameters $\phi$ and $\Tilde{\phi}$ of the encoder/decoder over the network's reconstruction loss, a useful latent representation of the data can be learnt by the autoencoder.
• Figure 2: As in the autoencoder shown in Figure \ref{['fig:litautoencoder']}, functions $\mathbf{\Tilde{h}}_{\Tilde{\phi}}$ and $\mathbf{h}_\phi$ encode/decode states $\mathbf{x}_k$ to/from latent states $\Tilde{\mathbf{x}}_k$. Linear operator $\mathbf{K}_\lambda$ is applied in the latent space, to propagate the latent state $\Tilde{\mathbf{x}}_k$ one time step forward into $\Tilde{\mathbf{x}}_{k+1}$. By optimizing the encoder/Koopman approximator/decoder's trainable parameters $\phi$, $\lambda$, and $\Tilde{\phi}$, over the standard autoencoder's loss function with additional linearity/forecasting terms, the Koopman autoencoder is able to learn a latent representation of the data in which the system's dynamics act linearly.
• Figure 3: An example plot of the KAE forecast's mean square error (MSE) against KAE EnKF’s frequency estimate. Data is generated for a discrete dynamical system, representing a simple sine wave with frequency $\theta = \pi$. Forecast for 10,000 states of this system are produced, with each forecast horizon being randomly selected from between 1 and 10 steps ahead. The resultant optimization landscape for the KAE EnKF's frequency estimate $\theta$ over the forecast's mean squared error, contains a strong global minimum at $\theta = \pi$ as expected, however also exhibits a further 9 local minima across the frequency's spectrum due to the use of 10 different forecast horizons in the optimization procedure.
• Figure 4: An example trajectory for the first dimension of the synthetic system's full state $\mathbf{x}_k$, with nonlinearity parameter $\nu = 3$, and both low $\sigma=0.05$ and high $\sigma=0.5$ levels of measurement noise, over the course of 1000 time steps.
• Figure 5: The eigenvalue modulus estimates,(top), argument estimates (middle), and mean squared 10-step ahead forecast error distributions (bottom), for latent state and full state KAE EnKF's, with low (left) and high (right) levels of measurement noise. The latent state KAE EnKF's parameter estimates and state forecasts are more stable and accurate than those of the full state KAE EnKF over all noise levels.