IEnSF: Iterative Ensemble Score Filter for Reducing Error in Posterior Score Estimation in Nonlinear Data Assimilation

TL;DR

This paper tackles nonlinear data assimilation by addressing structural bias in posterior score estimation within ensemble score filtering. It introduces the Iterative Ensemble Score Filter (IEnSF), which places an outer iterative loop around the reverse-time SDE and derives an exact posterior score under a Gaussian mixture prior, then develops practical, linearized approximations and a Kalman-type update for the likelihood score. A key innovation is the iterative refinement of the likelihood evaluation point $\bar{\boldsymbol{\mu}}_0^*(\boldsymbol{z}_t)$, coupled with observation-informed weighting and a derived time-scaling $\mathbf{J}(t)$, which together significantly reduce posterior-score error and bias. Numerical experiments across linear, nonlinear, and high-dimensional chaotic settings (including Lorenz-96) demonstrate improved posterior consistency, better handling of non-Gaussian posteriors, and competitive performance relative to LETKF, highlighting diffusion-based ensemble methods as scalable and accurate tools for uncertainty quantification in complex geophysical systems.

Abstract

The Ensemble Score Filter (EnSF) has emerged as a promising approach to leverage score-based diffusion models for solving high-dimensional and nonlinear data assimilation problems. While initial applications of EnSF to the Lorenz-96 model and the quasi-geostrophic system showed potential, the current method employs a heuristic weighted sum to combine the prior and the likelihood score functions. This introduces a structural error into the estimation of the posterior score function in the nonlinear setting. This work addresses this challenge by developing an iterative ensemble score filter (IEnSF) that applies an iterative algorithm as an outer loop around the reverse-time stochastic differential equation solver. When the state dynamics or the observation operator is nonlinear, the iterative algorithm can gradually reduce the posterior score estimation error by improving the accuracy of approximating the conditional expectation of the likelihood score function. The number of iterations required depends on the distance between the prior and posterior distributions and the nonlinearity of the observation operator. Numerical experiments demonstrate that the IEnSF algorithm substantially reduces the error in posterior score estimation in the nonlinear setting and thus improves the accuracy of tracking high-dimensional dynamical systems.

Figures (11)

• Figure 1: Posterior samples generated during the iterative refinement procedure for the linear observation model. The true posterior is Gaussian and obtained analytically from the Kalman update. Iterative updates progressively reduce the discrepancy between the samples and the true posterior.
• Figure 2: KL divergence and posterior samples for the linear observation model. Left: KL divergence between posterior samples and the true Gaussian posterior across iterations. Middle: Posterior samples from EnSF. Right: Posterior samples from IEnSF at iteration 5. The proposed IEnSF converges rapidly toward the true posterior, while EnSF retains a persistent bias.
• Figure 3: Non-Gaussian posterior sampling with a nonlinear observation operator. Results are shown for IEnSF (first four iterations), EnKF, particle filter, the original EnSF, and MCMC (ground truth). IEnSF captures the non-Gaussian posterior effectively even at the first iteration, since the initial reference posterior (based on the prior distribution) is already close to the true posterior.
• Figure 4: Bias correction in posterior sampling under a nonlinear observation operator. Compared methods include PF, EnKF, EnSF, and IEnSF. IEnSF reduces bias over iterations and converges toward the true posterior.
• Figure 5: KL divergence of posterior ensembles against the KF posterior for the harmonic oscillator problem. Each method uses 200 ensemble members with 10 repetitions. Error bars show the 10th and 90th percentiles. EnKF performs best due to its Gaussian assumption, while IEnSF matches PF closely. EnSF performs poorly due to limitations in its posterior score formulation.

Theorems & Definitions (17)

• Remark 3.2
• Proposition 3.3: The forward and reverse transitional probabilities
• Proposition 3.4: The prior score function
• Theorem 3.5: The exact posterior score
• Remark 3.6: Comparison with EKF
• Remark 3.7: Comparison with Laplace approximation
• Remark 3.8: Interpolation behavior of $\Bar{\boldsymbol{\mu}}_0^*(\boldsymbol{z}_t)$
• Lemma A.1: Distribution of the forward SDE
• proof
• Lemma A.2: Gradient of exponentiated functions