Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural Bayesian Filtering

Christopher Solinas, Radovan Haluska, David Sychrovsky, Finbarr Timbers, Nolan Bard, Michael Buro, Martin Schmid, Nathan R. Sturtevant, Michael Bowling

TL;DR

Neural Bayesian Filtering addresses the challenge of maintaining complex, multimodal belief states in partially observable settings by learning a latent belief-embedding that parameterizes a conditioned generative model. The method performs particle-like posterior updates in embedding space, enabling efficient sampling and density estimation with far fewer particles while accommodating non-stationary dynamics via known policies and environments. By combining a learnable embedding with a normalizing flow, NBF bridges classical filtering efficiency and the expressiveness of deep generative models, mitigating particle impoverishment. Empirical results across Gridworld, Goofspiel, and Triangulation demonstrate that NBF achieves competitive or superior accuracy with substantially reduced particle counts, illustrating its potential for scalable planning and estimation under uncertainty.

Abstract

We present Neural Bayesian Filtering (NBF), an algorithm for maintaining distributions over hidden states, called beliefs, in partially observable systems. NBF is trained to find a good latent representation of the beliefs induced by a task. It maps beliefs to fixed-length embedding vectors, which condition generative models for sampling. During filtering, particle-style updates compute posteriors in this embedding space using incoming observations and the environment's dynamics. NBF combines the computational efficiency of classical filters with the expressiveness of deep generative models - tracking rapidly shifting, multimodal beliefs while mitigating the risk of particle impoverishment. We validate NBF in state estimation tasks in three partially observable environments.

Neural Bayesian Filtering

TL;DR

Neural Bayesian Filtering addresses the challenge of maintaining complex, multimodal belief states in partially observable settings by learning a latent belief-embedding that parameterizes a conditioned generative model. The method performs particle-like posterior updates in embedding space, enabling efficient sampling and density estimation with far fewer particles while accommodating non-stationary dynamics via known policies and environments. By combining a learnable embedding with a normalizing flow, NBF bridges classical filtering efficiency and the expressiveness of deep generative models, mitigating particle impoverishment. Empirical results across Gridworld, Goofspiel, and Triangulation demonstrate that NBF achieves competitive or superior accuracy with substantially reduced particle counts, illustrating its potential for scalable planning and estimation under uncertainty.

Abstract

We present Neural Bayesian Filtering (NBF), an algorithm for maintaining distributions over hidden states, called beliefs, in partially observable systems. NBF is trained to find a good latent representation of the beliefs induced by a task. It maps beliefs to fixed-length embedding vectors, which condition generative models for sampling. During filtering, particle-style updates compute posteriors in this embedding space using incoming observations and the environment's dynamics. NBF combines the computational efficiency of classical filters with the expressiveness of deep generative models - tracking rapidly shifting, multimodal beliefs while mitigating the risk of particle impoverishment. We validate NBF in state estimation tasks in three partially observable environments.

Paper Structure

This paper contains 46 sections, 5 theorems, 22 equations, 8 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Main Contributions
  3. Belief State Embeddings
  4. A Flexible Parametric Framework For Filtering
  5. Background
  6. Notation
  7. Classical Filtering Algorithms
  8. Embedding Belief States
  9. Model Definition
  10. Embedding Function.
  11. Conditional Normalizing Flow.
  12. Discrete Belief States and Variational Dequantization.
  13. Illustrative Example: Donuts
  14. Filtering with Embeddings
  15. Convergence of NBF with a Perfect Model
  16. ...and 31 more sections

Key Result

Theorem 4.1

Assume $\epsilon$-global observation positivity of $(G, \pi)$ and a finite $X$ and $Y$. For any finite horizon $t_{\text{max}}$, belief state $p_t(x), t \leq t_{\text{max}}$, and any bounded function $\varphi:X \rightarrow \mathbb R$, let be the estimate of $\mathbb{E}_{p_t}[\varphi]$ computed by NBF with a perfect embedding model and $n$ particles. Then, as $n \rightarrow \infty$.

Figures (8)

  • Figure 1: Tracking an agent with an unknown starting position from observations about which direction the agent moved (with some probability of error) and whether or not it hit a wall. Colored cells indicate probabilities of possible agent positions.
  • Figure 2: Embedding the set of donut distributions in $\mathbb R^2$
  • Figure 3: Neural Bayesian Filtering generates particles from a belief embedding, simulates them in the environment, and embeds them with a weight proportional to the probability of $y$.
  • Figure 4: Jensen-Shannon divergence on fixed grids and policies (left to right: 5-2D, 8-2D, 5-3D, 8-3D). Training is repeated for 100 random seeds, with each model evaluated over 500 episodes. Shaded areas indicate $\pm$ 1 standard error on the average model performance.
  • Figure 5: Jensen-Shannon divergence on randomized grids and policies (left to right: 5-2D, 8-2D, 5-3D, 8-3D). Training is repeated for 100 random seeds, with each model evaluated over 500 episodes. Shaded areas indicates $\pm$ 1 standard error on the average model performance.
  • ...and 3 more figures

Theorems & Definitions (8)

  • Theorem 4.1: NBF Consistency
  • Corollary 4.2: NBF Convergence Rate
  • Lemma B.1: Strong Law for Self-Normalized Importance Sampling Estimators
  • proof
  • Theorem B.2: NBF Consistency
  • proof
  • Corollary B.3
  • proof