Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Limits of Learning Dynamical Systems

Tyrus Berry, Suddhasattwa Das

TL;DR

This analysis finds that forecast performance can depend on how well these other aspects of the dynamics are approximated, and examines whether an approximation of any one of these aspects of the dynamics could lead to an approximation of another facet.

Abstract

A dynamical system is a transformation of a phase space, and the transformation law is the primary means of defining as well as identifying the dynamical system. It is the object of focus of many learning techniques. Yet there are many secondary aspects of dynamical systems - invariant sets, the Koopman operator, and Markov approximations, which provide alternative objectives for learning techniques. Crucially, while many learning methods are focused on the transformation law, we find that forecast performance can depend on how well these other aspects of the dynamics are approximated. These different facets of a dynamical system correspond to objects in completely different spaces - namely interpolation spaces, compact Hausdorff sets, unitary operators and Markov operators respectively. Thus learning techniques targeting any of these four facets perform different kinds of approximations. We examine whether an approximation of any one of these aspects of the dynamics could lead to an approximation of another facet. Many connections and obstructions are brought to light in this analysis. Special focus is put on methods of learning of the primary feature - the dynamics law itself. The main question considered is the connection of learning this law with reconstructing the Koopman operator and the invariant set. The answers are tied to the ergodic and topological properties of the dynamics, and reveal how these properties determine the limits of forecasting techniques.

Limits of Learning Dynamical Systems

TL;DR

This analysis finds that forecast performance can depend on how well these other aspects of the dynamics are approximated, and examines whether an approximation of any one of these aspects of the dynamics could lead to an approximation of another facet.

Abstract

A dynamical system is a transformation of a phase space, and the transformation law is the primary means of defining as well as identifying the dynamical system. It is the object of focus of many learning techniques. Yet there are many secondary aspects of dynamical systems - invariant sets, the Koopman operator, and Markov approximations, which provide alternative objectives for learning techniques. Crucially, while many learning methods are focused on the transformation law, we find that forecast performance can depend on how well these other aspects of the dynamics are approximated. These different facets of a dynamical system correspond to objects in completely different spaces - namely interpolation spaces, compact Hausdorff sets, unitary operators and Markov operators respectively. Thus learning techniques targeting any of these four facets perform different kinds of approximations. We examine whether an approximation of any one of these aspects of the dynamics could lead to an approximation of another facet. Many connections and obstructions are brought to light in this analysis. Special focus is put on methods of learning of the primary feature - the dynamics law itself. The main question considered is the connection of learning this law with reconstructing the Koopman operator and the invariant set. The answers are tied to the ergodic and topological properties of the dynamics, and reveal how these properties determine the limits of forecasting techniques.
Paper Structure (35 sections, 6 theorems, 67 equations, 8 figures, 2 tables)

This paper contains 35 sections, 6 theorems, 67 equations, 8 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Data-driven discovery
  3. The invariant set
  4. Koopman operator
  5. Markov process
  6. Goal
  7. Outline
  8. Learning the dynamics law
  9. Embedding mechanism
  10. Feedback function
  11. The reconstructed system
  12. Hypothesis space
  13. Paradigm I : Invariant graphs
  14. II. Delay coordinates
  15. Connecting different descriptions
  16. ...and 20 more sections

Key Result

Proposition 2.1

\newlabelthm:kncd3l0 Let Assumption A:f hold, and $\phi:\Omega\to \mathbb{R}^d$ be a continuous map, and $g:\mathbb{R}^d\times \mathbb{R}^L \to \mathbb{R}^L$ be a $C^1$ map satisfying eqn:cv93m. Then

Figures (8)

  • Figure 1: Many features of a dynamical system. If a dynamical system generates data, there are many different aspects of the dynamics that one can try to recover from the data. These aspects are shown in green, and some common techniques in white.
  • Figure 1: Relating different aspects of a dynamical system. Figure \ref{['fig:outline1']} has been expanded by adding connections (yellow) between the four aspects (green) of dynamical systems.
  • Figure 1: Iteration error. The flowchart presents an analysis of the error resulting in an iterative forecasting of a dynamical system. The green boxes represent various conceptual dynamical systems needed to interpret the result. The white boxes mentions in brief the theoretical connections between these links. The target is a dynamical system shown in the top left corner. The main realization of Section \ref{['sec:predict']} and Theorem \ref{['thm:iterative']} is that the performance of the prediction can be expressed as the sum of two separate parts, as shown in yellow boxes above. The residual components can be described using purely by operator theoretic means, as an iteration of the Koopman operator $U$ on a subspace. The other component represents the fluctuations from this simple form. These fluctuations can be modeled by a special dynamical system called a perturbed matrix cocycle. The residual component can be studied well from the operator theoretic properties of $U$. The growth of the fluctuations can be understood well from the ergodic theory of matrix cocycles.
  • Figure 1: Performance of the two reconstruction techniques for (i) a quasiperiodic rotation on the torus $\mathbb{T}^2$ (bottom panels); and (ii) a Cartesian product of L63 with a simple harmonic oscillator (top panels). By Theorem \ref{['thm:direct']} (iv), if one has a proper embedding and a good approximation $\hat{w}$ of $w$ one can achieve arbitrarily small errors for the torus rotation, for all forecast times. This is supported by the fact that the direct methods for both the paradigms show errors of the order of $10^{-6}$. Since the torus rotation has all Lyapunov exponents zero, by Theorem \ref{['thm:iterative']} (ii) and Theorem \ref{['thm:lambda1']} (ii), the error from the iterative techniques should grow sub-exponentially, as supported by the figures. The system \ref{['eqn:L63Rot']} is a mixed spectrum system, i.e., the splitting in \ref{['eqn:def:L2split']} is non-trivial.
  • Figure 2: Gaps in learning techniques for dynamics. Figure \ref{['fig:outline1']} has been expanded by adding missing links (red) that should connect the four aspects (green) of dynamical systems.
  • ...and 3 more figures

Theorems & Definitions (6)

  • Proposition 2.1
  • Proposition 2.2
  • Theorem 4.1: Stability of reconstruction
  • Corollary 4.2
  • Theorem 5.1: Error from direct forecast
  • Theorem 5.2: Error from iterative forecast