Long-time accuracy of ensemble Kalman filters for chaotic and machine-learned dynamical systems

TL;DR

This work establishes rigorous long-time accuracy guarantees for ensemble Kalman filters (EnKF) applied to partially observed, high-dimensional, chaotic dynamical systems, including Navier–Stokes and Lorenz models. It proves two main results: (i) long-time accuracy of the square-root EnKF with inflation for the true dynamics, and (ii) long-time accuracy when the dynamics are replaced by machine-learned surrogate models, provided the surrogates have controlled short-term error in the unobserved components. The proofs leverage a mean-field Gaussian projected filter as an intermediate step and are complemented by mean-field surrogate analyses, yielding error bounds that scale with the observation noise level ε (and surrogate error δ) rather than the full state dimension. Numerical experiments on Lorenz-96 corroborate the theory, showing accurate long-time state estimation even with surrogates that are only short-term accurate, thereby supporting the practical use of ML surrogates in data assimilation for geophysical systems.

Abstract

Filtering is concerned with online estimation of the state of a dynamical system from partial and noisy observations. In applications where the state is high dimensional, ensemble Kalman filters are often the method of choice. This paper establishes long-time accuracy of ensemble Kalman filters. We introduce conditions on the dynamics and the observations under which the estimation error remains small in the long-time horizon. Our theory covers a wide class of partially-observed chaotic dynamical systems, which includes the Navier-Stokes equations and Lorenz models. In addition, we prove long-time accuracy of ensemble Kalman filters with surrogate dynamics, thus validating the use of machine-learned forecast models in ensemble data assimilation.

Figures (4)

• Figure 1: Average filter error over 50 Monte Carlo trials for decreasing observation noise levels, plotted with two standard errors (shaded).
• Figure 2: The structure of $\Psi^s$. The output channels of $\text{CNN}_1$ are divided into three groups of equal length, $\text{CNN}_1^{(1)}$, $\text{CNN}_1^{(2)}$, and $\text{CNN}_1^{(3)}$. The input channels to $\text{CNN}_2$ are a concatenation of $\text{CNN}_1^{(1)}$ and $(\text{CNN}_1^{(1)}\times \text{CNN}_1^{(2)})$, where the multiplication is point-wise.
• Figure 3: Visualization of the true state trajectory over time, the ensemble means, the observed data, and the difference between the true state and the ensemble means.
• Figure 4: The left plot displays the average filter error over 50 Monte Carlo trials plotted with two standard errors. The right plot displays the average surrogate model forecast error from an exactly known initial condition over 50 Monte Carlo trials plotted with two standard errors. This figure shows that long-time filter accuracy is possible with surrogate models that only give accurate short-term forecasts.

Theorems & Definitions (20)

• Theorem 2.2
• Remark 2.3
• Example 2.4
• Example 2.5
• Example 2.6
• Theorem 2.8
• Lemma 3.1
• Proof 1
• Theorem 3.2
• Proof 2