Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CGNSDE: Conditional Gaussian Neural Stochastic Differential Equation for Modeling Complex Systems and Data Assimilation

Chuanqi Chen, Nan Chen, Jin-Long Wu

TL;DR

CGNSDE introduces a hybrid modeling framework that combines causal-inference-derived, explainable physics-based components with neural-network augmentations to model complex dynamical systems while enabling analytic data assimilation. By enforcing a conditional Gaussian structure, the approach yields closed-form posterior updates for unobserved states, which are used to train the neural parts via a dedicated DA loss in addition to the usual forecast loss. Across chaotic, intermittent, and high-dimensional settings, CGNSDE demonstrates improved state estimation and uncertainty quantification, including accurate handling of extreme events and long-term statistics. The work highlights the value of integrating causal discovery, physics-based modeling, and neural networks to achieve both interpretability and predictive accuracy in data-assimilation-enabled dynamical modeling.

Abstract

A new knowledge-based and machine learning hybrid modeling approach, called conditional Gaussian neural stochastic differential equation (CGNSDE), is developed to facilitate modeling complex dynamical systems and implementing analytic formulae of the associated data assimilation (DA). In contrast to the standard neural network predictive models, the CGNSDE is designed to effectively tackle both forward prediction tasks and inverse state estimation problems. The CGNSDE starts by exploiting a systematic causal inference via information theory to build a simple knowledge-based nonlinear model that nevertheless captures as much explainable physics as possible. Then, neural networks are supplemented to the knowledge-based model in a specific way, which not only characterizes the remaining features that are challenging to model with simple forms but also advances the use of analytic formulae to efficiently compute the nonlinear DA solution. These analytic formulae are used as an additional computationally affordable loss to train the neural networks that directly improve the DA accuracy. This DA loss function promotes the CGNSDE to capture the interactions between state variables and thus advances its modeling skills. With the DA loss, the CGNSDE is more capable of estimating extreme events and quantifying the associated uncertainty. Furthermore, crucial physical properties in many complex systems, such as the translate-invariant local dependence of state variables, can significantly simplify the neural network structures and facilitate the CGNSDE to be applied to high-dimensional systems. Numerical experiments based on chaotic systems with intermittency and strong non-Gaussian features indicate that the CGNSDE outperforms knowledge-based regression models, and the DA loss further enhances the modeling skills of the CGNSDE.

CGNSDE: Conditional Gaussian Neural Stochastic Differential Equation for Modeling Complex Systems and Data Assimilation

TL;DR

CGNSDE introduces a hybrid modeling framework that combines causal-inference-derived, explainable physics-based components with neural-network augmentations to model complex dynamical systems while enabling analytic data assimilation. By enforcing a conditional Gaussian structure, the approach yields closed-form posterior updates for unobserved states, which are used to train the neural parts via a dedicated DA loss in addition to the usual forecast loss. Across chaotic, intermittent, and high-dimensional settings, CGNSDE demonstrates improved state estimation and uncertainty quantification, including accurate handling of extreme events and long-term statistics. The work highlights the value of integrating causal discovery, physics-based modeling, and neural networks to achieve both interpretability and predictive accuracy in data-assimilation-enabled dynamical modeling.

Abstract

A new knowledge-based and machine learning hybrid modeling approach, called conditional Gaussian neural stochastic differential equation (CGNSDE), is developed to facilitate modeling complex dynamical systems and implementing analytic formulae of the associated data assimilation (DA). In contrast to the standard neural network predictive models, the CGNSDE is designed to effectively tackle both forward prediction tasks and inverse state estimation problems. The CGNSDE starts by exploiting a systematic causal inference via information theory to build a simple knowledge-based nonlinear model that nevertheless captures as much explainable physics as possible. Then, neural networks are supplemented to the knowledge-based model in a specific way, which not only characterizes the remaining features that are challenging to model with simple forms but also advances the use of analytic formulae to efficiently compute the nonlinear DA solution. These analytic formulae are used as an additional computationally affordable loss to train the neural networks that directly improve the DA accuracy. This DA loss function promotes the CGNSDE to capture the interactions between state variables and thus advances its modeling skills. With the DA loss, the CGNSDE is more capable of estimating extreme events and quantifying the associated uncertainty. Furthermore, crucial physical properties in many complex systems, such as the translate-invariant local dependence of state variables, can significantly simplify the neural network structures and facilitate the CGNSDE to be applied to high-dimensional systems. Numerical experiments based on chaotic systems with intermittency and strong non-Gaussian features indicate that the CGNSDE outperforms knowledge-based regression models, and the DA loss further enhances the modeling skills of the CGNSDE.
Paper Structure (23 sections, 28 equations, 11 figures, 7 tables)

This paper contains 23 sections, 28 equations, 11 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Conditional Gaussian Neural Stochastic Differential Equation (CGNSDE)
  3. The conditional Gaussian nonlinear system
  4. CGNSDE for modeling complex systems and DA
  5. Procedure of Developing a CGNSDE
  6. Determining the knowledge-based components by causal inference
  7. Determining the neural network components
  8. The short-term forecast loss
  9. The DA loss
  10. Training the CGNSDE with both the forecast and the DA losses
  11. Numerical Results
  12. Overview
  13. The Lorenz 84 model: a low-order chaotic system
  14. The projected stochastic Burgers–Sivashinsky equation: a highly nonlinear system with intermittency
  15. The Lorenz 96 model: A moderate-dimensional chaotic system
  16. ...and 8 more sections

Figures (11)

  • Figure 4.1: Model trajectories and the associated statistics of the true Lorenz 84 system. Panel (a): time series of each variable. Panel (b): the probability density function (PDF). Panel (c): the auto-correlation function (ACF). It should be noted that the PDFs and ACFs are estimated from much longer time series than the ones presented in Panel (a).
  • Figure 4.2: DA results of Lorenz 84 system from the closed analytic formulae in \ref{['eq:CGNS_Filter']} for true system and three models. The uncertainties are indicated by the grey colored regions, which correspond to two standard deviation from the posterior mean.
  • Figure 4.3: One long-term realization of the CGNSDE model trained with both forecast and DA losses for the Lorenz 84 system. Panel (a): time series of different variables. Panel (b) and (c): the associated PDFs and ACFs.
  • Figure 4.4: One simulation of the projected stochastic Burgers–Sivashinsky equation \ref{['eq:PSBSE']}. Panel (a): time series of each state variable. Panel (b): the PDFs. Panel (c): the ACFs. It should be noted that the PDFs and ACFs are estimated from much longer simulations than the one presented in Panel (a).
  • Figure 4.5: DA results of the projected stochastic Burgers–Sivashinsky equation from EnKBF for true system and closed analytic formulae in \ref{['eq:CGNS_Filter']} for three models. The uncertainties are indicated by the grey colored regions, which correspond to two standard deviations from the posterior mean.
  • ...and 6 more figures