Data Assimilation With An Integral-Form Ensemble Square-Root Filter

TL;DR

This work tackles the computational bottleneck of high-dimensional data assimilation with localization by introducing InFo-ESRF, an integral-form ensemble square-root filter. By reformulating the perturbation update as a sum of symmetric linear-system solves and employing a heavy-tailed quadrature to approximate an integral, InFo-ESRF avoids explicit matrix square-roots and exploits preconditioned Krylov methods for fast, accurate updates. Across synthetic Gaussian and multi-layer Lorenz-type experiments, InFo-ESRF demonstrates competitive or superior accuracy and reduced cost relative to existing localized square-root filters, with strong parallelizability and scalability. The approach promises practical impact for large-scale weather prediction and other high-dimensional DA problems where localization and ensemble limitations arise.

Abstract

Geoscientific applications of ensemble Kalman filters face several computational challenges arising from the high dimensionality of the forecast covariance matrix, particularly when this matrix incorporates localization. For square-root filters, updating the perturbations of the ensemble members from their mean is an especially challenging step, one which generally requires approximations that introduce a trade-off between accuracy and computational cost. This paper describes an ensemble square-root filter which achieves a favorable trade-off between these factors by discretizing an integral representation of the Kalman filter update equations, and in doing so, avoids a direct evaluation of the matrix square-root in the perturbation update stage. This algorithm, which we call InFo-ESRF ("Integral-Form Ensemble Square-Root Filter"), is parallelizable and uses a preconditioned Krylov method to update perturbations to a high degree of accuracy. Through numerical experiments with both a Gaussian forecast model and a multi-layer Lorenz-type system, we demonstrate that InFo-ESRF is competitive or superior to several existing localized square-root filters in terms of accuracy and cost.

Figures (6)

• Figure 1: Left: a $100 \times 100$ covariance matrix ${\mathbf{\Sigma}}_{xx}$. Middle: an ensemble estimate $\widetilde{{\mathbf{\Sigma}}}_{xx} = {\mathbf{Z}}{\mathbf{Z}}^\mathrm{T}$, where ${\mathbf{Z}}$ is derived from 20 independent samples from ${\mathcal{N}}({\vec{0}},\, {\mathbf{\Sigma}}_{xx})$. Right: a localized estimate $\widehat{{\mathbf{\Sigma}}}_{xx} = {\mathbf{L}} \circ ({\mathbf{Z}}{\mathbf{Z}}^\mathrm{T})$, where ${\mathbf{L}}(i,j) = {\mathcal{G}}(|i - j|)$ and ${\mathcal{G}}$ is the Gaspari-Cohn localization function gaspari_cohn of radius 20.
• Figure 2: Left: leading entries of the eigenvalue spectrum for ${\mathbf{\Sigma}}_{xx}$ defined in \ref{['eqn:single_cycle_covar']}. Right: forecast and analysis variances for the observing system given by \ref{['eqn:single_cycle_obsop', 'eqn:single_cycle_obserr_covar']}, over a subset of the state vector. The local minima in analysis variance correspond to peaks in the weighting functions that define each observation channel.
• Figure 3: Results of the synthetic forecast experiment. Horizontal error bars for 95% confidence are too narrow to visually resolve in this experiment, and are therefore not shown. Vertical error bars are similarly narrow for all filters except the serial ESRF and GETKF with randomized SVD-based modulation, for which they stretch into the negative.
• Figure 4: Top: a typical state vector for the model "atmosphere." Middle: a Hovmöller diagram showing the time evolution of a single column. Bottom: weighting functions for the 5 channels observed for a each measured column.
• Figure 5: Results of the cycled data assimilation experiment. For the augmentation-based filters, $k$ is the ratio of the augmented ensemble size to the original ensemble size. For InFo-ESRF, $k$ is the number of quadrature nodes.

Theorems & Definitions (8)

• Theorem 1
• proof
• Theorem 2
• proof
• Theorem 3
• proof
• Proposition 1
• proof