Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Koopman-based Deep Learning for Nonlinear System Estimation

Zexin Sun, Mingyu Chen, John Baillieul

TL;DR

This paper presents a novel data-driven linear estimator based on Koopman operator theory to extract meaningful finite-dimensional representations of complex non-linear systems.

Abstract

Nonlinear differential equations are encountered as models of fluid flow, spiking neurons, and many other systems of interest in the real world. Common features of these systems are that their behaviors are difficult to describe exactly and invariably unmodeled dynamics present challenges in making precise predictions. In this paper, we present a novel data-driven linear estimator based on Koopman operator theory to extract meaningful finite-dimensional representations of complex non-linear systems. The Koopman model is used together with deep reinforcement networks that learn the optimal stepwise actions to predict future states of nonlinear systems. Our estimator is also adaptive to a diffeomorphic transformation of the estimated nonlinear system, which enables it to compute optimal state estimates without re-learning.

Koopman-based Deep Learning for Nonlinear System Estimation

TL;DR

This paper presents a novel data-driven linear estimator based on Koopman operator theory to extract meaningful finite-dimensional representations of complex non-linear systems.

Abstract

Nonlinear differential equations are encountered as models of fluid flow, spiking neurons, and many other systems of interest in the real world. Common features of these systems are that their behaviors are difficult to describe exactly and invariably unmodeled dynamics present challenges in making precise predictions. In this paper, we present a novel data-driven linear estimator based on Koopman operator theory to extract meaningful finite-dimensional representations of complex non-linear systems. The Koopman model is used together with deep reinforcement networks that learn the optimal stepwise actions to predict future states of nonlinear systems. Our estimator is also adaptive to a diffeomorphic transformation of the estimated nonlinear system, which enables it to compute optimal state estimates without re-learning.
Paper Structure (8 sections, 2 theorems, 22 equations, 3 figures, 1 table, 1 algorithm)

This paper contains 8 sections, 2 theorems, 22 equations, 3 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Deep Reinforcement Learning-based Estimator
  4. EDMD-based Linear Approximation
  5. Deep Reinforcement Learning-based Estimator
  6. Transfer Learning-based Optimal Policy
  7. Simulation and Analysis
  8. Conclusion

Key Result

Lemma 1

The loss function in Algorithm alg:general_DDPG is non-increasing.

Figures (3)

  • Figure 1: The left figure exhibits the evolved reward during training using the EDMD merged with DDPG and only DDPG algorithm respectively. Figures on the right presented illustrate the performance of our estimator in approximating three distinct trajectories, each originating from different initial points. The arrows indicate that the estimator is initialized with the same starting points as the respective trajectories at the beginning.
  • Figure 2: The figure depicts the estimation performance by employing a transferred optimal policy in contrast to a newly trained estimator (\ref{['eq:e_linear']}) based on 20 episodes on a diffeomorphic nonlinear system.
  • Figure 3: The figure exhibits the performance of our approach on the Van der Pol equation and compares the stand-alone RL approach with hybrid combined EDMD plus RL.

Theorems & Definitions (6)

  • Definition 1
  • Example 1
  • Lemma 1
  • proof
  • Theorem 1
  • proof