Neural Dynamical Operator: Continuous Spatial-Temporal Model with Gradient-Based and Derivative-Free Optimization Methods

TL;DR

A data-driven modeling framework called neural dynamical operator that is continuous in both space and time is presented and it is shown that the proposed model can better predict long-term statistics via the hybrid optimization scheme with a combined use of short-term and long-term data.

Abstract

Data-driven modeling techniques have been explored in the spatial-temporal modeling of complex dynamical systems for many engineering applications. However, a systematic approach is still lacking to leverage the information from different types of data, e.g., with different spatial and temporal resolutions, and the combined use of short-term trajectories and long-term statistics. In this work, we build on the recent progress of neural operator and present a data-driven modeling framework called neural dynamical operator that is continuous in both space and time. A key feature of the neural dynamical operator is the resolution-invariance with respect to both spatial and temporal discretizations, without demanding abundant training data in different temporal resolutions. To improve the long-term performance of the calibrated model, we further propose a hybrid optimization scheme that leverages both gradient-based and derivative-free optimization methods and efficiently trains on both short-term time series and long-term statistics. We investigate the performance of the neural dynamical operator with three numerical examples, including the viscous Burgers' equation, the Navier-Stokes equations, and the Kuramoto-Sivashinsky equation. The results confirm the resolution-invariance of the proposed modeling framework and also demonstrate stable long-term simulations with only short-term time series data. In addition, we show that the proposed model can better predict long-term statistics via the hybrid optimization scheme with a combined use of short-term and long-term data.

Figures (22)

• Figure 2.1: Schematic diagram of neural dynamical operator (based on Navier--Stokes equations). The dynamics of the system for the current state are approximated by neural operator, then the future states are evaluated with an ODE solver along with time given any initial state. The neural dynamical operator $\Tilde{\mathcal{G}}$ is trained by minimizing the Loss $L_s$ with gradient-based optimization.
• Figure 2.2: Schematic diagram of neural dynamical operator with hybrid optimization scheme (based on Navier--Stokes equations). To better generalize the model by utilizing both short-term and long-term data, the neural dynamical operator $\Tilde{\mathcal{G}}$ is trained by the hybrid optimization scheme which will iteratively update parameters by stochastic gradient descent (SGD) method to minimize short-term states loss $L_s$ and by derivative-free method (EKI) to minimize long-term statistics loss $L_l$. The short-term system evolution in $[t_0, t_{N_s}]$ corresponds to Fig. \ref{['fig:SchematicDiagram1']}.
• Figure 3.1: The spatial-temporal solutions of viscous Burgers' equation. Left column: true system. Middle column: trained models from three different resolutions with the same test data resolution ($dx=1/1024, dt=0.05$). Right column: errors of the solutions simulated based on the trained models. The index in the three trained models corresponds to the resolution settings in Table \ref{['tab:VBE_resolution']}.
• Figure 3.2: Solution profiles of viscous Burgers' equation for the true system and the model ones with different initial conditions from test data. The model is trained in a coarse resolution ($dx=1/64, dt=0.5$) and tested on a finer resolution ($dx=1/1024, dt=0.05$).
• Figure 3.3: Energy spectrum of the true system and the model ones with different test initial conditions (in rows) and at different times (in columns). The index in the three trained models corresponds to the resolution settings in Table \ref{['tab:VBE_resolution']}.