Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Convolutional Bayesian Filtering

Wenhan Cao, Shiqi Liu, Chang Liu, Zeyu He, Stephen S. -T. Yau, Shengbo Eben Li

TL;DR

This work extends Bayesian filtering by introducing convolutional conditional probability, transforming transition and output probabilities into convolution-like forms to explicitly address model mismatch. The core idea reduces to standard Bayes filtering in the Dirac-delta limit, while allowing inequality-based kernels to model mismatch, non-Gaussianities, and outliers. Analytical results show Gaussian cases reduce to robust Kalman filters with inflated noise covariances, and non-Gaussian cases are tractable via exponential density rescaling tied to information bottleneck theory. Empirical studies across linear and nonlinear models, including Wiener velocity, sequence forecasting, and a gas-phase reactor, demonstrate that convolutional filters (ConvKF/ConvEKF/ConvUKF/ConvPF) consistently outperform traditional filters under mismatch and heavy-tailed noise, offering practical robustness with minimal structural changes to existing algorithms.

Abstract

Bayesian filtering serves as the mainstream framework of state estimation in dynamic systems. Its standard version utilizes total probability rule and Bayes' law alternatively, where how to define and compute conditional probability is critical to state distribution inference. Previously, the conditional probability is assumed to be exactly known, which represents a measure of the occurrence probability of one event, given the second event. In this paper, we find that by adding an additional event that stipulates an inequality condition, we can transform the conditional probability into a special integration that is analogous to convolution. Based on this transformation, we show that both transition probability and output probability can be generalized to convolutional forms, resulting in a more general filtering framework that we call convolutional Bayesian filtering. This new framework encompasses standard Bayesian filtering as a special case when the distance metric of the inequality condition is selected as Dirac delta function. It also allows for a more nuanced consideration of model mismatch by choosing different types of inequality conditions. For instance, when the distance metric is defined in a distributional sense, the transition probability and output probability can be approximated by simply rescaling them into fractional powers. Under this framework, a robust version of Kalman filter can be constructed by only altering the noise covariance matrix, while maintaining the conjugate nature of Gaussian distributions. Finally, we exemplify the effectiveness of our approach by reshaping classic filtering algorithms into convolutional versions, including Kalman filter, extended Kalman filter, unscented Kalman filter and particle filter.

Convolutional Bayesian Filtering

TL;DR

This work extends Bayesian filtering by introducing convolutional conditional probability, transforming transition and output probabilities into convolution-like forms to explicitly address model mismatch. The core idea reduces to standard Bayes filtering in the Dirac-delta limit, while allowing inequality-based kernels to model mismatch, non-Gaussianities, and outliers. Analytical results show Gaussian cases reduce to robust Kalman filters with inflated noise covariances, and non-Gaussian cases are tractable via exponential density rescaling tied to information bottleneck theory. Empirical studies across linear and nonlinear models, including Wiener velocity, sequence forecasting, and a gas-phase reactor, demonstrate that convolutional filters (ConvKF/ConvEKF/ConvUKF/ConvPF) consistently outperform traditional filters under mismatch and heavy-tailed noise, offering practical robustness with minimal structural changes to existing algorithms.

Abstract

Bayesian filtering serves as the mainstream framework of state estimation in dynamic systems. Its standard version utilizes total probability rule and Bayes' law alternatively, where how to define and compute conditional probability is critical to state distribution inference. Previously, the conditional probability is assumed to be exactly known, which represents a measure of the occurrence probability of one event, given the second event. In this paper, we find that by adding an additional event that stipulates an inequality condition, we can transform the conditional probability into a special integration that is analogous to convolution. Based on this transformation, we show that both transition probability and output probability can be generalized to convolutional forms, resulting in a more general filtering framework that we call convolutional Bayesian filtering. This new framework encompasses standard Bayesian filtering as a special case when the distance metric of the inequality condition is selected as Dirac delta function. It also allows for a more nuanced consideration of model mismatch by choosing different types of inequality conditions. For instance, when the distance metric is defined in a distributional sense, the transition probability and output probability can be approximated by simply rescaling them into fractional powers. Under this framework, a robust version of Kalman filter can be constructed by only altering the noise covariance matrix, while maintaining the conjugate nature of Gaussian distributions. Finally, we exemplify the effectiveness of our approach by reshaping classic filtering algorithms into convolutional versions, including Kalman filter, extended Kalman filter, unscented Kalman filter and particle filter.
Paper Structure (14 sections, 5 theorems, 46 equations, 6 figures, 1 algorithm)

This paper contains 14 sections, 5 theorems, 46 equations, 6 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Convolutional Conditional Probability
  3. Convolutional Bayesian Filtering
  4. Uncertain Hidden Markov Model with Model Mismatch
  5. Filtering Algorithm
  6. Analytical Form in Gaussian Cases
  7. Approximation via Exponential Density Rescaling
  8. Exponential Density Rescaling
  9. Connection with Information Bottleneck Theory
  10. Simulations
  11. Linear Wiener Velocity Model
  12. Sequence Forecasting System
  13. Isothermal Gas-phase Reactor Model
  14. Conclusion and Discussion

Key Result

Proposition 1

The convolutional conditional probability satisfies

Figures (6)

  • Figure 1: (a) $\mathbf{y}$ and $\mathbf{z}$ are identical ($\mathbf{y}=\mathbf{z}$). (b) The distance between $\mathbf{y}$ and $\mathbf{z}$ is bounded ($d(\mathbf{y},\mathbf{z}) \leq \mathbf{r}$).
  • Figure 2: Illustration of uncertain HMM with model mismatch. The nominal transition probability in HMM projects the real state in the previous time, denoted as $\mathbf{x}_{t-1}$, to a model-predicted virtual state $\bar{\mathbf{x}}_t$. However, due to model mismatch, $\mathbf{x}_{t-1}$ transitions to the real state $x_t$ in the physical world. The transition from $\bar{\mathbf{x}}_t$ to $\mathbf{x}_t$ is inaccessible; instead, their distance $d(\mathbf{x}_t, \bar{\mathbf{x}}_t)$ are bounded by a threshold random variable $\mathbf{r}_x$, i.e., $d(\mathbf{x}_t, \bar{\mathbf{x}}_t) \leq \mathbf{r}_x$. Analogously, $d(\mathbf{y}_t, \bar{\mathbf{y}}_t) \leq \mathbf{r}_y$ reflects the measurement model mismatch.
  • Figure 3: (a) Illustration of information bottleneck. (b) Markov chain of the information bottleneck. Note that $G$ represents the data generator for $\mathbf{y}_t$ and $\bar{\mathbf{y}}_t$.
  • Figure 4: Box plot of RMSE for KF, HuberKF and ConvKF. The black square " $\blacksquare$ " represents the average RMSE.
  • Figure 5: Box plot of RMSE for UKF, EKF, HuberUKF, IEKF, ConvEKF and ConvUKF.
  • ...and 1 more figures

Theorems & Definitions (12)

  • Definition 1: Convolutional Conditional Probability
  • Proposition 1
  • Remark 1
  • Proposition 2: Limiting Property
  • Remark 2
  • Remark 3
  • Theorem 1: Convolutional Bayesian filtering
  • Remark 4
  • Corollary 1
  • Remark 5
  • ...and 2 more