Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A universal reproducing kernel Hilbert space for learning nonlinear systems operators

Mircea Lazar

TL;DR

This work addresses learning nonlinear operators $G(\mathbf{u})(x)$ that map input trajectories to output trajectories for discrete-time systems with initial states. It constructs a universal learning framework based on a product reproducing kernel Hilbert space $\mathcal{H}(k_\otimes,\mathcal{U}\times\mathcal{X})$ with $k_\otimes=(k_u k_x)$, enabling a universal interpolant through Gram matrix inverses and reflecting a kernelized view of radial-basis universal approximation. The main theoretical contributions show that, for positive definite kernels $k_u,k_x$, the product RKHS is dense and complete in the space of nonlinear system operators under reasonable conditions, and it provides a finite-data interpolation scheme $G_{\mathcal{H}}^*$ that exactly fits training data. The approach demonstrates favorable scalability compared to standard RKHS methods by exploiting a Kronecker-structured Gram matrix, as illustrated by a Van der Pol example where the product RKHS handles many more data points with reduced computational burden. Overall, the framework offers a principled, universal, and scalable tool for operator learning in nonlinear dynamics with potential links to Koopman theory and data-enabled predictive control.

Abstract

In this work, we consider the problem of learning nonlinear operators that correspond to discrete-time nonlinear dynamical systems with inputs. Given an initial state and a finite input trajectory, such operators yield a finite output trajectory compatible with the system dynamics. Inspired by the universal approximation theorem of operators tailored to radial basis functions neural networks, we construct a class of kernel functions as the product of kernel functions in the space of input trajectories and initial states, respectively. We prove that for positive definite kernel functions, the resulting product reproducing kernel Hilbert space is dense and even complete in the space of nonlinear systems operators, under suitable assumptions. This provides a universal kernel-functions-based framework for learning nonlinear systems operators, which is intuitive and easy to apply to general nonlinear systems.

A universal reproducing kernel Hilbert space for learning nonlinear systems operators

TL;DR

This work addresses learning nonlinear operators that map input trajectories to output trajectories for discrete-time systems with initial states. It constructs a universal learning framework based on a product reproducing kernel Hilbert space with , enabling a universal interpolant through Gram matrix inverses and reflecting a kernelized view of radial-basis universal approximation. The main theoretical contributions show that, for positive definite kernels , the product RKHS is dense and complete in the space of nonlinear system operators under reasonable conditions, and it provides a finite-data interpolation scheme that exactly fits training data. The approach demonstrates favorable scalability compared to standard RKHS methods by exploiting a Kronecker-structured Gram matrix, as illustrated by a Van der Pol example where the product RKHS handles many more data points with reduced computational burden. Overall, the framework offers a principled, universal, and scalable tool for operator learning in nonlinear dynamics with potential links to Koopman theory and data-enabled predictive control.

Abstract

In this work, we consider the problem of learning nonlinear operators that correspond to discrete-time nonlinear dynamical systems with inputs. Given an initial state and a finite input trajectory, such operators yield a finite output trajectory compatible with the system dynamics. Inspired by the universal approximation theorem of operators tailored to radial basis functions neural networks, we construct a class of kernel functions as the product of kernel functions in the space of input trajectories and initial states, respectively. We prove that for positive definite kernel functions, the resulting product reproducing kernel Hilbert space is dense and even complete in the space of nonlinear systems operators, under suitable assumptions. This provides a universal kernel-functions-based framework for learning nonlinear systems operators, which is intuitive and easy to apply to general nonlinear systems.

Paper Structure

This paper contains 9 sections, 2 theorems, 25 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Preliminary definitions and results
  3. Universal approximation of operators based on radial basis functions neural networks
  4. Reproducing kernel Hilbert spaces
  5. Problem formulation
  6. Product reproducing kernel Hilbert spaces
  7. Illustrative example
  8. Conclusions
  9. Acknowledgements

Key Result

theorem thmcountertheorem

Chen_RBF Suppose that $g\in C(\mathbb{R})\cap S'(\mathbb{R})$ is not an even polynomial, $\mathbb{U}\subseteq \mathcal{U}$, $\mathbb{X}\subseteq\mathcal{X}$ are two compact sets in $\mathcal{U}$ and $\mathcal{X}$, respectively and $G$ is a nonlinear continuous operator, which maps $\mathbb{U}$ into for all $\mathbf{u}\in \mathbb{U}$, all $x\in\mathbb{X}$ and all $q=1,\ldots,p(N+1)$.

Figures (4)

  • Figure 1: Output trajectories used for learning the operator in the product RKHS.
  • Figure 2: Validation of the product kernel operator predictions for several initial conditions not part of the training data and a random input signal.
  • Figure 3: Validation of the standard kernel mapping predictions for the same initial conditions and random input signal used to validate the product kernel operator.
  • Figure 4: RMS for the prediction of $x_1$ for each time step $i=1,\ldots,10$.

Theorems & Definitions (9)

  • theorem thmcountertheorem
  • definition thmcounterdefinition
  • definition thmcounterdefinition
  • definition thmcounterdefinition
  • definition thmcounterdefinition
  • definition thmcounterdefinition
  • theorem thmcountertheorem
  • proof
  • remark thmcounterremark