Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Variational Sequential Monte Carlo for High-Dimensional Observations

Wessel L. van Nierop, Nir Shlezinger, Ruud J. G. van Sloun

TL;DR

This work tackles state estimation for nonlinear systems with high-dimensional observations by introducing a differentiable particle filter whose proposal and transition are learned end-to-end via an unsupervised variational SMC objective. The approach uses neural networks to parameterize a Gaussian Mixture Model proposal conditioned on past states and current observations, combined with differentiable resampling to train without ground-truth states. Experiments on the Lorenz attractor with high-dimensional image observations show that the method improves tracking accuracy and yields a more faithful posterior distribution, as reflected in ELBO-based evaluations, outperforming EKF, BPF, and a supervised baseline. The results demonstrate that unsupervised learning of proposals in particle filters enhances robustness to observation noise and partial visibility, with potential impact on real-world high-dimensional filtering tasks.

Abstract

Sequential Monte Carlo (SMC), or particle filtering, is widely used in nonlinear state-space systems, but its performance often suffers from poorly approximated proposal and state-transition distributions. This work introduces a differentiable particle filter that leverages the unsupervised variational SMC objective to parameterize the proposal and transition distributions with a neural network, designed to learn from high-dimensional observations. Experimental results demonstrate that our approach outperforms established baselines in tracking the challenging Lorenz attractor from high-dimensional and partial observations. Furthermore, an evidence lower bound based evaluation indicates that our method offers a more accurate representation of the posterior distribution.

Deep Variational Sequential Monte Carlo for High-Dimensional Observations

TL;DR

This work tackles state estimation for nonlinear systems with high-dimensional observations by introducing a differentiable particle filter whose proposal and transition are learned end-to-end via an unsupervised variational SMC objective. The approach uses neural networks to parameterize a Gaussian Mixture Model proposal conditioned on past states and current observations, combined with differentiable resampling to train without ground-truth states. Experiments on the Lorenz attractor with high-dimensional image observations show that the method improves tracking accuracy and yields a more faithful posterior distribution, as reflected in ELBO-based evaluations, outperforming EKF, BPF, and a supervised baseline. The results demonstrate that unsupervised learning of proposals in particle filters enhances robustness to observation noise and partial visibility, with potential impact on real-world high-dimensional filtering tasks.

Abstract

Sequential Monte Carlo (SMC), or particle filtering, is widely used in nonlinear state-space systems, but its performance often suffers from poorly approximated proposal and state-transition distributions. This work introduces a differentiable particle filter that leverages the unsupervised variational SMC objective to parameterize the proposal and transition distributions with a neural network, designed to learn from high-dimensional observations. Experimental results demonstrate that our approach outperforms established baselines in tracking the challenging Lorenz attractor from high-dimensional and partial observations. Furthermore, an evidence lower bound based evaluation indicates that our method offers a more accurate representation of the posterior distribution.
Paper Structure (13 sections, 7 equations, 5 figures, 1 algorithm)

This paper contains 13 sections, 7 equations, 5 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Signal model
  4. Particle filter
  5. Methods
  6. Proposal learning
  7. Networks
  8. Experiments
  9. Lorenz attractor
  10. Baselines
  11. Metrics
  12. Results
  13. Conclusion

Figures (5)

  • Figure 1: Illustration demonstrating the proposed method of using a parameterized proposal distribution in particle filtering.
  • Figure 2: Example images of the Lorenz attractor (in the same position) using various noise levels with standard deviation $\sigma$ (top) and randomly dropped observations where $P$ represents the proportion of observations (bottom).
  • Figure 3: Tracking error of the DPF compared to the baseline methods for varying noise levels. The tracking error is computed using the Euclidean distance of the predicted location to the ground truth location. The error bars depict the 95% confidence interval calculated using 20 different seeds.
  • Figure 4: Tracking error of the DPF compared to the baseline methods for various levels of partial observations. The tracking error is computed using the Euclidean distance of the predicted location to the ground truth location. The error bars depict the 95% confidence interval calculated using 10 different seeds.
  • Figure 5: Decomposition of the ELBO for the DPF and the baselines for various levels of partial observations. The error bars depict the 95% confidence interval calculated using 10 different seeds.