Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Score Filter Enhanced Data Assimilation Framework for Data-Driven Dynamical Systems

Jingqiao Tang, Ryan Bausback, Feng Bao, Guannan Zhang, Phuoc-Toan Huynh

Abstract

We introduce a score-filter-enhanced data assimilation framework designed to reduce predictive uncertainty in machine learning (ML) models for data-driven dynamical system forecasting. Machine learning serves as an efficient numerical model for predicting dynamical systems. However, even with sufficient data, model uncertainty remains and accumulates over time, causing the long-term performance of ML models to deteriorate. To overcome this difficulty, we integrate data assimilation techniques into the training process to iteratively refine the model predictions by incorporating observational information. Specifically, we apply the Ensemble Score Filter (EnSF), a generative AI-based training-free diffusion model approach, for solving the data assimilation problem in high-dimensional nonlinear complex systems. This leads to a hybrid data assimilation-training framework that combines ML with EnSF to improve long-term predictive performance. We shall demonstrate that EnSF-enhanced ML can effectively reduce predictive uncertainty in ML-based Lorenz-96 system prediction and the Korteweg-De Vries (KdV) equation prediction.

A Score Filter Enhanced Data Assimilation Framework for Data-Driven Dynamical Systems

Abstract

We introduce a score-filter-enhanced data assimilation framework designed to reduce predictive uncertainty in machine learning (ML) models for data-driven dynamical system forecasting. Machine learning serves as an efficient numerical model for predicting dynamical systems. However, even with sufficient data, model uncertainty remains and accumulates over time, causing the long-term performance of ML models to deteriorate. To overcome this difficulty, we integrate data assimilation techniques into the training process to iteratively refine the model predictions by incorporating observational information. Specifically, we apply the Ensemble Score Filter (EnSF), a generative AI-based training-free diffusion model approach, for solving the data assimilation problem in high-dimensional nonlinear complex systems. This leads to a hybrid data assimilation-training framework that combines ML with EnSF to improve long-term predictive performance. We shall demonstrate that EnSF-enhanced ML can effectively reduce predictive uncertainty in ML-based Lorenz-96 system prediction and the Korteweg-De Vries (KdV) equation prediction.
Paper Structure (20 sections, 2 theorems, 29 equations, 16 figures, 1 table, 1 algorithm)

This paper contains 20 sections, 2 theorems, 29 equations, 16 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Overview of the Underlying Machine Learning Models
  3. Long-Short Term Memory
  4. Deep Operator Networks for Operator Learning
  5. Recurrent Deep Operator Network
  6. Data Assimilation Framework and the Ensemble Score Filter
  7. Ensemble Score Filter
  8. Integrating Data Assimilation with Machine Learning Models
  9. Numerical Experiments
  10. Lorenz-96 Model
  11. Training configuration for the LSTM model
  12. Experimental setup for data assimilation
  13. Numerical results for a sufficiently trained AI model
  14. Numerical results for insufficiently trained LSTM models
  15. Korteweg-De Vries (KdV) equation
  16. ...and 5 more sections

Key Result

Theorem 1

Suppose $\sigma$ is a continuous non-polynomial function, $X$ is a Banach space, $K_1 \subset X$, $K_2 \subset \mathbb{R}^d$ are compact sets, $V$ is a compact subset of $C(K_1)$, and $G$ is an operator mapping $V$ to $C(K_2)$. Then for $\epsilon > 0$, there are positive integers $n$, $p$, $m$, cons holds for all $u \in V$ and $y \in K_2$. Here, $C(K)$ is the Banach space of all continuous functio

Figures (16)

  • Figure 1: A block diagram of an LSTM cell, which takes recurrent input $h^{t-1}$, gated self-loop input $c^{t-1}$, and current state input vector $x^t$. These then flow through the forget gate $f^t$, the dual input gates $g^t$ and $\tilde{c}^t$, and output gate $o^t$ when making predictions for sequential and iterative data yu2019review.
  • Figure 2: The traditional approach of numerically solving the governing equations versus data-driven approach on predicting the spatio-temporal models.
  • Figure 3: LSTM training loss for clean and noisy datasets. The model trained on clean data achieves low training error with stable convergence, while training on noisy data leads to less stable convergence and higher residual error.
  • Figure 4: Comparison of LSTM-SSP and LSTM-LTP predictions. A well-trained LSTM achieves accurate short-term predictions, but its performance degrades for long-term prediction when true states are no longer provided as conditioning inputs.
  • Figure 5: Schematic of the data assimilation procedure. After the initial 20 simulation steps, observations are available for 15 out of every 20 time steps.
  • ...and 11 more figures

Theorems & Definitions (2)

  • Theorem 1: Universal Approximation Theorem for Operators Chen1995
  • Theorem 2: Generalized Universal Approximation Theorem for Operators, Lu2021