Small Ensemble-based Data Assimilation: A Machine Learning-Enhanced Data Assimilation Method with Limited Ensemble Size

TL;DR

The paper tackles the computational burden of ensemble-based data assimilation by addressing the accuracy loss with small ensembles. It introduces EnKF-FCNN, which couples a traditional Ensemble Kalman Filter with a fully connected neural network to learn a correction term $\Delta{\boldsymbol{s}}_{a,j}$ that compensates for limited ensemble size, using $\boldsymbol{S}^{\mathfrak{N}}_{a,j}$, $\bar{\boldsymbol{s}}_{m,j}$, and $\bar{\boldsymbol{s}}_{a,j-1}$ as inputs to predict $\Delta{\boldsymbol{s}}_{a,j} = \bar{\boldsymbol{s}}^{\mathcal{N}}_{a,j} - \bar{\boldsymbol{s}}^{\mathfrak{N}}_{a,j}$. The approach is validated on Lorenz-63, Lorenz-96, and nonlinear ocean wave simulations (PFL-EnKF), showing substantial accuracy gains over standard EnKF with the same small ensemble and negligible additional computational cost. Lorenz-63 achieves about a one-order-of-magnitude reduction in the error metric, Lorenz-96 shows a notable but smaller improvement due to higher nonlinearity, and ocean-wave tests demonstrate that EnKF-FCNN with ${\mathfrak{N}}=20$ can match ${\mathcal{N}}=100}$ with >50% error reduction. The results indicate the method’s adaptability across models and DA schemes, offering a practical path to robust, scalable data assimilation with limited ensemble resources.

Abstract

Ensemble-based data assimilation (DA) methods have become increasingly popular due to their inherent ability to address nonlinear dynamic problems. However, these methods often face a trade-off between analysis accuracy and computational efficiency, as larger ensemble sizes required for higher accuracy also lead to greater computational cost. In this study, we propose a novel machine learning-based data assimilation approach that combines the traditional ensemble Kalman filter (EnKF) with a fully connected neural network (FCNN). Specifically, our method uses a relatively small ensemble size to generate preliminary yet suboptimal analysis states via EnKF. A FCNN is then employed to learn and predict correction terms for these states, thereby mitigating the performance degradation induced by the limited ensemble size. We evaluate the performance of our proposed EnKF-FCNN method through numerical experiments involving Lorenz systems and nonlinear ocean wave field simulations. The results consistently demonstrate that the new method achieves higher accuracy than traditional EnKF with the same ensemble size, while incurring negligible additional computational cost. Moreover, the EnKF-FCNN method is adaptable to diverse applications through coupling with different models and the use of alternative ensemble-based DA methods.

Figures (12)

• Figure 1: Schematic illustration of EnKF (reproduced with modifications from wang2021phase). The size of ellipses represents the level of uncertainty. We use the notation $\boldsymbol{s}^{(n)}_{*,j}$ to represent the $n_{th}$ ($n=1,2,\dots N$) ensemble member state at time $t=t_j$, $j=0,1,2\dots$, with $*=m,f,a$ for measurement, forecast, and analysis.
• Figure 2: Schematic illustration of the EnKF-FCNN coupled algorithm. For the purpose of brevity, only the ensemble mean is shown, omitting individual members.
• Figure 3: Analysis results obtained with the traditional EnKF using $\mathcal{N}=100$ () and $\mathfrak{N}=7$ (), as well as the true solution (-- -- --), for Lorenz-63
• Figure 4: Analysis results obtained with the traditional EnKF using $\mathcal{N}=100$ () and $\mathfrak{N}=7$ (), as well as the true solution (-- -- --), for Lorenz-96
• Figure 5: Analysis results given by the EnKF-FCNN algorithm with $\mathfrak{N}=7$(-- -- --) and traditional EnKF with $\mathcal{N}=100$ () for Lorenz-63