Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Online Gaussian Process State-Space Model: Learning and Planning for Partially Observable Dynamical Systems

Soon-Seo Park, Young-Jin Park, Youngjae Min, Han-Lim Choi

TL;DR

An online learning method of Gaussian process state-space model (GP-SSM) that incorporates stochastic variational inference (VI) and online VI with novel formulation is proposed, which mitigates the computational complexity without catastrophic forgetting and also support adaptation to changes in a system and/or real environments.

Abstract

This paper proposes an online learning method of Gaussian process state-space model (GP-SSM). GP-SSM is a probabilistic representation learning scheme that represents unknown state transition and/or measurement models as Gaussian processes (GPs). While the majority of prior literature on learning of GP-SSM are focused on processing a given set of time series data, data may arrive and accumulate sequentially over time in most dynamical systems. Storing all such sequential data and updating the model over entire data incur large amount of computational resources in space and time. To overcome this difficulty, we propose a practical method, termed \textit{onlineGPSSM}, that incorporates stochastic variational inference (VI) and online VI with novel formulation. The proposed method mitigates the computational complexity without catastrophic forgetting and also support adaptation to changes in a system and/or a real environments. Furthermore, we present application of onlineGPSSM into the reinforcement learning (RL) of partially observable dynamical systems by integrating onlineGPSSM with Bayesian filtering and trajectory optimization algorithms. Numerical examples are presented to demonstrate applicability of the proposed method.

Online Gaussian Process State-Space Model: Learning and Planning for Partially Observable Dynamical Systems

TL;DR

An online learning method of Gaussian process state-space model (GP-SSM) that incorporates stochastic variational inference (VI) and online VI with novel formulation is proposed, which mitigates the computational complexity without catastrophic forgetting and also support adaptation to changes in a system and/or real environments.

Abstract

This paper proposes an online learning method of Gaussian process state-space model (GP-SSM). GP-SSM is a probabilistic representation learning scheme that represents unknown state transition and/or measurement models as Gaussian processes (GPs). While the majority of prior literature on learning of GP-SSM are focused on processing a given set of time series data, data may arrive and accumulate sequentially over time in most dynamical systems. Storing all such sequential data and updating the model over entire data incur large amount of computational resources in space and time. To overcome this difficulty, we propose a practical method, termed \textit{onlineGPSSM}, that incorporates stochastic variational inference (VI) and online VI with novel formulation. The proposed method mitigates the computational complexity without catastrophic forgetting and also support adaptation to changes in a system and/or a real environments. Furthermore, we present application of onlineGPSSM into the reinforcement learning (RL) of partially observable dynamical systems by integrating onlineGPSSM with Bayesian filtering and trajectory optimization algorithms. Numerical examples are presented to demonstrate applicability of the proposed method.

Paper Structure

This paper contains 16 sections, 51 equations, 9 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Problem Formulation
  4. Stochastic Variational Inference in GP-SSM
  5. Evidence Lower Bound (ELBO) and Stochastic ELBO evaluation
  6. Main Result: Online Learning of GP-SSM
  7. Application: GP-SSM-based Reinforcement Learning Framework
  8. State Prediction and Estimation via Bayesian Filtering
  9. Probabilistic Trajectory Optimization
  10. Approximate Belief Dynamics and Cost Function
  11. Control Policy
  12. Summary of Algorithm
  13. Numerical Simulations
  14. A simple linear system
  15. Loitering of fixed-wing UAV in a wind field
  16. ...and 1 more sections

Figures (9)

  • Figure 1: Graphical model of GP-SSM. White and gray nodes represent latent and observed variables, respectively. The thick lines indicate sets of fully connected variables, which are jointly Gaussian under the GP prior.
  • Figure 2: Proposed online learning and planning framework.
  • Figure 3: Reconstruction results after 5-step online learning.
  • Figure 4: Comparison of RMSE w.r.t. training time.
  • Figure 5: Comparison of mean log-likelihood w.r.t. training time.
  • ...and 4 more figures