Kernel Operator-Theoretic Bayesian Filter for Nonlinear Dynamical Systems

TL;DR

This paper presents several practical implementations to obtain a finite-dimensional approximation of the functional Bayesian filter (FBF), and demonstrates that this practical approach can obtain accurate results and outperform finite-dimensional Koopman decomposition.

Abstract

Motivated by the surge of interest in Koopman operator theory, we propose a machine-learning alternative based on a functional Bayesian perspective for operator-theoretic modeling of unknown, data-driven, nonlinear dynamical systems. This formulation is directly done in an infinite-dimensional space of linear operators or Hilbert space with universal approximation property. The theory of reproducing kernel Hilbert space (RKHS) allows the lifting of nonlinear dynamics to a potentially infinite-dimensional space via linear embeddings, where a general nonlinear function is represented as a set of linear functions or operators in the functional space. This allows us to apply classical linear Bayesian methods such as the Kalman filter directly in the Hilbert space, yielding nonlinear solutions in the original input space. This kernel perspective on the Koopman operator offers two compelling advantages. First, the Hilbert space can be constructed deterministically, agnostic to the nonlinear dynamics. The Gaussian kernel is universal, approximating uniformly an arbitrary continuous target function over any compact domain. Second, Bayesian filter is an adaptive, linear minimum-variance algorithm, allowing the system to update the Koopman operator and continuously track the changes across an extended period of time, ideally suited for modern data-driven applications such as real-time machine learning using streaming data. In this paper, we present several practical implementations to obtain a finite-dimensional approximation of the functional Bayesian filter (FBF). Due to the rapid decay of the Gaussian kernel, excellent approximation is obtained with a small dimension. We demonstrate that this practical approach can obtain accurate results and outperform finite-dimensional Koopman decomposition.

Figures (9)

• Figure 1: In a discrete-time nonlinear dynamical system, the nonlinearity generates a nonlinear manifold on which the time-series $\textbf{x}_i$ resides. DMD approximates the evolution on a nonlinear manifold using a least-squares fit linear dynamical system. The Koopman operator linearizes the space the data is embedded. Similarly, in the theory of the RKHS, the input data is mapped to a higher-dimensional functional space where a linear operator $\textbf{F}$ advances the states of the system, approximating the general nonlinear transition function $f(\cdot)$ in the input space.
• Figure 2: General state-space model for dynamical system.
• Figure 3: Recurrent network trained using (a) Square-Root Cubature Kalman Filter and RNN (b) Functional Bayesian Filter using the tensor-product kernel (c) Explicit Hilbert space FBF using a sum kernel.
• Figure 4: Ensemble-averaged Mean-Squared Error (MSE) over 50 runs vs. number of batch iterations (each training iteration consists of a 100-sample sequence with random starting point).
• Figure 5: Short-term future state reconstruction errors of the NLS dynamics using a standard DMD approximation ${\bf g}_{\hbox{\tiny DMD}} ({\bf x})$, the NLS motivated observables ${\bf g}_1({\bf x})$, the generic quadratic observables ${\bf g}_2({\bf x})$, the Gaussian quadrature observable, and the explicit Hilbert space FBF using GQ observable: (a) the NLS reconstruction errors for DMD and expFBF, (b) the MSE summed across space for the four reconstructions.

Theorems & Definitions (2)

• Theorem 1: Bochner, 1932Bochner1959
• Definition 1: Koopman Operator