Adaptive tempering schedules with approximative intermediate measures for filtering problems

TL;DR

This paper tackles the challenge of non-Gaussian posteriors in high-dimensional data assimilation by proposing an adaptive tempering framework that combines Gaussian-approximative filters with nonparametric, approximative consistent filters through intermediate measures. The authors introduce two tempering criteria to decide when to apply tempered updates and demonstrate the approach on toy models including Langevin dynamics, Lorenz systems, and shallow-water equations, showing improved RMSE and robustness relative to fixed-tempering or single-filter setups. The core contributions are the adaptive tempering schedule, the use of approximative intermediate measures, and the empirical evidence that IQR-based tempering provides robust improvements across filter families. This framework offers a practical pathway to more reliable uncertainty quantification in nonlinear, high-dimensional filtering problems with potential applicability to real-world atmospheric and oceanic data assimilation tasks.

Abstract

Data assimilation algorithms integrate prior information from numerical model simulations with observed data. Ensemble-based filters, regarded as state-of-the-art, are widely employed for large-scale estimation tasks in disciplines such as geoscience and meteorology. Despite their inability to produce the true posterior distribution for nonlinear systems, their robustness and capacity for state tracking are noteworthy. In contrast, Particle filters yield the correct distribution in the ensemble limit but require substantially larger ensemble sizes than ensemble-based filters to maintain stability in higher-dimensional spaces. It is essential to transcend traditional Gaussian assumptions to achieve realistic quantification of uncertainties. One approach involves the hybridisation of filters, facilitated by tempering, to harness the complementary strengths of different filters. A new adaptive tempering method is proposed to tune the underlying schedule, aiming to systematically surpass the performance previously achieved. Although promising numerical results for certain filter combinations in toy examples exist in the literature, the tuning of hyperparameters presents a considerable challenge. A deeper understanding of these interactions is crucial for practical applications.

Figures (7)

• Figure 1: Illustration of a typical box plot. The red dot denotes the observation.
• Figure 2: Illustration of localization, Blue are the grid points, green the currently updated point, orange the available observation and red the selected obs
• Figure 3: Visualisation of the prior and posterior distributions as well as the truth and observation over two time steps (from left to right), highlighting the notable tail switching dynamics between time steps.
• Figure 4: The box plots include the observations and the ensemble state at the first state of the Lorenz 63 over a range of 50 assimilating cycles. The upper and lower diagram depicts the statistics computed by the plain ETPF and the tempered (IQR-criterion) version respectively.
• Figure 5: The box plots include the observations and the ensemble state at the first grid point of the Lorenz 96 discretization over a range of 50 assimilating cycles. The upper and lower diagram depicts the statistics computed by the plain ETPF and the tempered (IQR-criterion) version respectively.