Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sequential Estimation of Gaussian Process-based Deep State-Space Models

Yuhao Liu, Marzieh Ajirak, Petar Djuric

TL;DR

This work tackles sequential estimation in GP-based state-space and deep state-space models where the governing functions are unknown. It combines random feature-based Gaussian processes with particle filtering and Bayesian linear regression to perform online inference, and introduces an ensemble of feature sets to reduce variance and improve robustness. The main contributions are a PF-based online inference framework for GP-SSMs and GP-DSSMs, an ensemble learning strategy, and demonstrations on synthetic and real datasets showing accurate latent-state tracking and competitive performance. The approach enables scalable, uncertainty-aware forecasting in nonlinear, nonstationary systems, though deeper models require more particles and features to maintain accuracy, pointing to future work in variational feature selection and efficient sampling.

Abstract

We consider the problem of sequential estimation of the unknowns of state-space and deep state-space models that include estimation of functions and latent processes of the models. The proposed approach relies on Gaussian and deep Gaussian processes that are implemented via random feature-based Gaussian processes. In these models, we have two sets of unknowns, highly nonlinear unknowns (the values of the latent processes) and conditionally linear unknowns (the constant parameters of the random feature-based Gaussian processes). We present a method based on particle filtering where the parameters of the random feature-based Gaussian processes are integrated out in obtaining the predictive density of the states and do not need particles. We also propose an ensemble version of the method, with each member of the ensemble having its own set of features. With several experiments, we show that the method can track the latent processes up to a scale and rotation.

Sequential Estimation of Gaussian Process-based Deep State-Space Models

TL;DR

This work tackles sequential estimation in GP-based state-space and deep state-space models where the governing functions are unknown. It combines random feature-based Gaussian processes with particle filtering and Bayesian linear regression to perform online inference, and introduces an ensemble of feature sets to reduce variance and improve robustness. The main contributions are a PF-based online inference framework for GP-SSMs and GP-DSSMs, an ensemble learning strategy, and demonstrations on synthetic and real datasets showing accurate latent-state tracking and competitive performance. The approach enables scalable, uncertainty-aware forecasting in nonlinear, nonstationary systems, though deeper models require more particles and features to maintain accuracy, pointing to future work in variational feature selection and efficient sampling.

Abstract

We consider the problem of sequential estimation of the unknowns of state-space and deep state-space models that include estimation of functions and latent processes of the models. The proposed approach relies on Gaussian and deep Gaussian processes that are implemented via random feature-based Gaussian processes. In these models, we have two sets of unknowns, highly nonlinear unknowns (the values of the latent processes) and conditionally linear unknowns (the constant parameters of the random feature-based Gaussian processes). We present a method based on particle filtering where the parameters of the random feature-based Gaussian processes are integrated out in obtaining the predictive density of the states and do not need particles. We also propose an ensemble version of the method, with each member of the ensemble having its own set of features. With several experiments, we show that the method can track the latent processes up to a scale and rotation.
Paper Structure (25 sections, 42 equations, 13 figures, 2 tables, 1 algorithm)

This paper contains 25 sections, 42 equations, 13 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Gaussian Processes
  4. Random Feature-Based Gaussian Processes
  5. Bayesian Linear Regression
  6. Particle Filtering
  7. Gaussian Process State Space Model
  8. Propagation of the particles
  9. Updating of the joint posteriors of $({\hbox{\boldmath $\eta$}}^{(m),[d]}, \sigma_u^{{(m),[d]}^2})$
  10. Updating of the joint posteriors of $(\pmb{\theta}^{(m),[d]}, \sigma_v^{{(m),[d]}^2})$
  11. Weight computation of particles and estimation of ${\bf x}_t$
  12. Ensemble Learning
  13. Ensemble Estimates of the States
  14. Keep and Drop
  15. Gaussian Process-based Deep State-Space Models
  16. ...and 10 more sections

Figures (13)

  • Figure 1: A generic diagram of an SSM.
  • Figure 2: A generic diagram of a deep SSM with $L$ layers.
  • Figure 3: Point estimates and 95% confidence region of $\widehat{x}_t^{[1]}$ and its true values $x_t^{[1]}$ for the last 100 samples.
  • Figure 4: Estimates of $\widehat{x}_t^{[2]}$ under 20 runs and its true values $x_t^{[2]}$.
  • Figure 5: Pairs of estimated ($\widehat{\textbf{x}}_t$) and actual ($\textbf{x}_t$) latent states before SVD.
  • ...and 8 more figures