Exact nonlinear state estimation

TL;DR

This work addresses biases in traditional data assimilation caused by Gaussian assumptions by introducing the Conjugate Transform Filter (CTF), a nonlinear state-space formulation that propagates Gaussian priors through invertible, learnable maps to produce non-Gaussian but tractable posteriors. An ensemble counterpart, ECTF, leverages an EnKF-like update in a latent Gaussian space, enabling principled, non-Gaussian updates while preserving Bayesian consistency. Theoretical results show that CTF generalizes the Kalman filter (and its affine variants) and yields exact nonlinear filtering in the transformed space, while numerical experiments demonstrate that ECTF outperforms the standard EnKF—especially when observations are accurate and state variables are strongly nonlinear or highly correlated. Overall, the framework provides a bridge between traditional DA and AI-based, nonparametric approaches, with practical implications for high-dimensional Earth system modeling and beyond.

Abstract

The majority of data assimilation (DA) methods in the geosciences are based on Gaussian assumptions. While these assumptions facilitate efficient algorithms, they cause analysis biases and subsequent forecast degradations. Non-parametric, particle-based DA algorithms have superior accuracy, but their application to high-dimensional models still poses operational challenges. Drawing inspiration from recent advances in the field of generative artificial intelligence (AI), this article introduces a new nonlinear estimation theory which attempts to bridge the existing gap in DA methodology. Specifically, a Conjugate Transform Filter (CTF) is derived and shown to generalize the celebrated Kalman filter to arbitrarily non-Gaussian distributions. The new filter has several desirable properties, such as its ability to preserve statistical relationships in the prior state and convergence to highly accurate observations. An ensemble approximation of the new theory (ECTF) is also presented and validated using idealized statistical experiments that feature bounded quantities with non-Gaussian distributions, a prevalent challenge in Earth system models. Results from these experiments indicate that the greatest benefits from ECTF occur when observation errors are small relative to the forecast uncertainty and when state variables exhibit strong nonlinear dependencies. Ultimately, the new filtering theory offers exciting avenues for improving conventional DA algorithms through their principled integration with AI techniques.

Figures (6)

• Figure 1: A probabilistic representation of the state estimation problem. The general state-space model (\ref{['eq:SSM']}) is integrated into a hidden Markov model, with horizontal and vertical arrows corresponding to the state transition dynamics and the observation generation process, respectively.
• Figure 2: A schematic illustration of the Conjugate Transform Filter (CTF). Starting with a Gaussian distribution (gray shading) at time $t_0$, the state is propagated to time $t_1$ when the first observations ${\mathrm{y}}_1$ become available. Owing to the nonlinear dynamics (thin blue curves), the initial state distribution deforms and results in a non-Gaussian prior, which needs to be approximated through a suitable choice of ${\mathbf{f}}_{{\uptheta}_1}$. The observations ${\mathrm{y}}_1$ are then used by CTF to update the prior, but only the latent Gaussian parameters $\{\upmu_{1|0}, {\mathbf{\Sigma}}_{1|0}\}$ are modified, whereas the values of ${\uptheta}_1$ remain unchanged. The same process is repeated until the current filtering time $t_k$ when the last batch of observations ${\mathrm{y}}_k$ arrives. Notice that while the non-Gaussian state distributions are allowed to evolve over time, the conjugate nature of the update ensures that the prior and posterior belong to the same distribution family.
• Figure 3: Using the prior ensemble $\{{\mathrm{Z}}_k^{f,i}\}_{i=1}^{N_e}$ to fit the parameters of the non-Gaussian prior pdf $p({\mathrm{z}}_k | {\mathrm{y}}_{1:k-1}) = {\mathbf{f}}_{{\uptheta}_k\, \sharp} \, \phi \left[{\mathrm{z}}_k ;\upmu_{k|k-1},{\mathbf{\Sigma}}_{k|k-1} \right]$ at time $k$.
• Figure 4: Ensemble filtering skill in terms of the Jensen-Shannon (JS) divergence. Panel (a) shows the scaled JS values for EnKF, whereas (b) and (c) -- percentage changes in this metric for QCEF-LR and ECTF. For each parameter pair $\{ \rho,r \}$, the ensemble filtering performance is averaged over 1000 independent DA trials. The black crosses in (b) indicate parameter values where the mean JS differences with EnKF are not statistically significant at the 95% confidence level according to a two-sided t-test. The lack of black crosses in (c) indicates that all ECTF-EnKF differences are statistically significant.
• Figure 5: A closer examination of the $\{\rho=0.99,r=0.01\}$ parameter regime. Panels (a)--(d) display the median performance of EnKF (black), QCEF-LR (blue) and ECTF (red) as a function of the innovation values $d_k$ in (\ref{['eq:innovation']}) and with respect to all 4 skill metrics described in Section \ref{['subsec:exps_skill_metrics']}. The shading around each curve indicates the interquartile range (IQR) calculated from the multiple trial DA experiments (hard to distinguish for ECTF due to the small variability in its skill).