Flow-based Bayesian filtering for high-dimensional nonlinear stochastic dynamical systems

TL;DR

The paper tackles nonlinear Bayesian filtering in high-dimensional stochastic systems where explicit dynamical models are unavailable. It introduces the flow-based Bayesian filter (FBF), which uses normalizing flows to map states and observations into a latent space where a latent linear Gaussian state-space model enables Kalman-like updates, ensuring Gaussian latent posteriors at each step. Training optimizes a likelihood-based objective over both state-transition and observation likelihoods, while online filtering performs efficient Gaussian updates and decodes samples back to the original space via invertible transforms. Empirical results on a 2D nonlinear system, high-dimensional Lorenz-96, and a stochastic advection–diffusion model show that FBF achieves superior accuracy and uncertainty quantification with favorable compute costs compared to particle filters, RKN, and CNF-DPF. Overall, the method offers a scalable, data-driven approach for accurate high-dimensional filtering without requiring explicit system dynamics or observation models.

Abstract

Bayesian filtering for high-dimensional nonlinear stochastic dynamical systems is a fundamental yet challenging problem in many fields of science and engineering. Existing methods face significant obstacles: Gaussian-based filters struggle with non-Gaussian distributions, while sequential Monte Carlo methods are computationally intensive and prone to particle degeneracy in high dimensions. Although generative models in machine learning have made significant progress in modeling high-dimensional non-Gaussian distributions, their inefficiency in online updating limits their applicability to filtering problems. To address these challenges, we propose a flow-based Bayesian filter (FBF) that integrates normalizing flows to construct a novel latent linear state-space model with Gaussian filtering distributions. This framework facilitates efficient density estimation and sampling using invertible transformations provided by normalizing flows, and it enables the construction of filters in a data-driven manner, without requiring prior knowledge of system dynamics or observation models. Numerical experiments demonstrate the superior accuracy and efficiency of FBF.

Figures (10)

• Figure 1: Schematic of FBF. The dashed arrows represent the execution of traditional Bayesian filtering, while the solid arrows indicate the direction of the proposed FBF.
• Figure 2: Comparison of performance using different methods in \ref{['subsection:sinusoidal-example']}, where bars represent the mean and error bars indicate the standard deviation calculated over 200 test cases. (a) RMSE; (b) MMD; (c): CRPS.
• Figure 3: The estimated filtering distribution of the inferred state for a given measurement at time step $k = 50$ by four methods. In each figure, the diagonal displays the histograms of state vector, while the lower left triangle and upper right triangle areas show the pairwise scatter diagram and density estimation of the same state vector, respectively.
• Figure 4: Comparison of performance using different methods associated with varying dimensionality of states, where the bars represent the mean and the error bars indicate the standard deviation across 200 test cases. (a) RMSE; (b) MMD; (c) CRPS.
• Figure 5: Visualization of mean and uncertainty of the estimated posterior for a test measurement of $10$-dimensional Lorenz-96 systems. For clarity, we present the final 500 steps of the test trajectory of length $K_{\mathrm{test}} = 1500$.

Theorems & Definitions (2)

• Remark 3.1
• Remark 3.2