Non-intrusive structural-preserving sequential data assimilation

TL;DR

This paper tackles data assimilation for systems where governing equations are unknown by integrating structure-preserving learning with structure-informed updates. The authors introduce NSSDA, which pairs an entropy-stable conservative flux form neural network (ESCFN) surrogate for forecasting with a structurally informed ensemble transform Kalman filter (SETKF) for analysis, enabling non-intrusive, physics-consistent state estimation from a single noisy trajectory. Across 1D dam-break (shallow water) and Shu–Osher (Euler) problems, NSSDA yields significantly improved predictive accuracy and stability, particularly near shocks and discontinuities, compared to standard Neural ODEs or ETKF variants. The work demonstrates the practical potential of combining physical structure with local statistical structure in data assimilation, with implications for geophysical and climate applications where governing models are incomplete or unavailable.

Abstract

Data assimilation (DA) methods combine model predictions with observational data to improve state estimation in dynamical systems, inspiring their increasingly prominent role in geophysical and climate applications. Classical DA methods assume that the governing equations modeling the dynamics are known, which is unlikely for most real world applications. Machine learning (ML) provides a flexible alternative by learning surrogate models directly from data, but standard ML methods struggle in noisy and data-scarce environments, where meaningful extrapolation requires incorporating physical constraints. Recent advances in structure-preserving ML architectures, such as the development of the entropy-stable conservative flux form network (ESCFN), highlight the critical role of physical structure in improving learning stability and accuracy for unknown systems of conservation laws. Structural information has also been shown to improve DA performance. Gradient-based measures of spatial variability, in particular, can help refine ensemble updates in discontinuous systems. Motivated by both of these recent innovations, this investigation proposes a new non-intrusive, structure-preserving sequential data assimilation (NSSDA) framework that leverages structure at both the forecast and analysis stages. We use the ESCFN to construct a surrogate model to preserve physical laws during forecasting, and a structurally informed ensemble transform Kalman filter (SETKF) to embed local statistical structure into the assimilation step. Our method operates in a highly constrained environment, using only a single noisy trajectory for both training and assimilation. Numerical experiments where the unknown dynamics correspond respectively to the shallow water and Euler equations demonstrate significantly improved predictive accuracy.

Figures (7)

• Figure 1: The dam break problem solution at time $t = 1$. (left) Predictions using ${\boldsymbol\Psi}_{NN}^{\boldsymbol\theta}$ and ${\boldsymbol\Psi}_{CFN}^{\boldsymbol\theta}$; (middle) the posterior ensemble means using ETKF; (right) posterior ensemble means using SETKF. (top) height $h$; (bottom) velocity $u$.
• Figure 2: Comparison of relative $\ell^2$ errors for height $h$ (left) and velocity $u$ (right). The solid line shows errors for predictions using ${\boldsymbol\Psi}_{CFN}^{\boldsymbol\theta}$. The dashed and dotted lines show the errors from ETKF and SETKF post-processing, respectively.
• Figure 3: (left) ESCFN predictions with $L_{train} = 10,20,40$; (middle) corresponding posterior ensemble means using ETKF; and (right) corresponding posterior ensemble means using SETKF at $t = 1$. Observation variances: (top) $\sigma^2 = 0.1$ and (bottom) $\sigma^2 = 0.2$.
• Figure 4: (left) relative $\ell^2$ errors for velocity $u$ of ESCFN predictions for $L_{train} = 10,20,40$; (middle) corresponding posterior relative $\ell^2$ error using ETKF, and (right) corresponding posterior relative $\ell^2$ error using SETKF over time $t \in \left[ 0,1 \right]$. Observation variances (top) $\sigma^2 = 0.1$ and (bottom) $\sigma^2 = 0.2$.
• Figure 5: (left) relative $\ell^2$ errors for height $h$ of ESCFN predictions for $L_{train} = 10,20,40$; (middle) corresponding posterior relative $\ell^2$ error using ETKF, and (right) corresponding posterior relative $\ell^2$ error using SETKF over time $t \in \left[ 0,1 \right]$. Observation variances (top) $\sigma^2 = 0.1$ and (bottom) $\sigma^2 = 0.2$.

Theorems & Definitions (5)

• Remark 3.1
• Remark 3.2
• Remark 3.3
• Remark 4.1
• Remark 4.2