Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Derivative estimation by RKHS regularization for learning dynamics from time-series data

Hailong Guo, Haibo Li

TL;DR

This work tackles learning dynamical systems from noisy time-series by jointly estimating derivatives and trajectories through a vector-valued RKHS framework. It develops an integral-form representer theorem that reduces the derivative estimation to a finite-dimensional linear system, enabling efficient regularization parameter selection via the L-curve. By embedding the dynamics within a vRKHS, the underlying vector field $oldsymbol{f}$ is recovered as a linear regularization problem, offering a flexible nonparametric alternative to dictionary-based methods such as SINDy. Through extensive numerical experiments on classic nonlinear systems, the approach demonstrates improved robustness to noise, accurate derivative estimation, and effective learning and prediction of dynamical behavior. The method provides a practical, kernel-based pipeline for denoising and dynamical-system identification without requiring hand-crafted dictionaries, with potential extensions to scalable kernel techniques for large datasets.

Abstract

Learning the governing equations from time-series data has gained increasing attention due to its potential to extract useful dynamics from real-world data. Despite significant progress, it becomes challenging in the presence of noise, especially when derivatives need to be calculated. To reduce the effect of noise, we propose a method that simultaneously fits both the derivative and trajectory from noisy time-series data. Our approach formulates derivative estimation as an inverse problem involving integral operators within the forward model, and estimates the derivative function by solving a regularization problem in a vector-valued reproducing kernel Hilbert space (vRKHS). We derive an integral-form representer theorem, which enables the computation of the regularized solution by solving a finite-dimensional problem and facilitates efficiently estimating the optimal regularization parameter. By embedding the dynamics within a vRKHS and utilizing the fitted derivative and trajectory, we can recover the underlying dynamics from noisy data by solving a linear regularization problem. Several numerical experiments are conducted to validate the effectiveness and efficiency of our method.

Derivative estimation by RKHS regularization for learning dynamics from time-series data

TL;DR

This work tackles learning dynamical systems from noisy time-series by jointly estimating derivatives and trajectories through a vector-valued RKHS framework. It develops an integral-form representer theorem that reduces the derivative estimation to a finite-dimensional linear system, enabling efficient regularization parameter selection via the L-curve. By embedding the dynamics within a vRKHS, the underlying vector field is recovered as a linear regularization problem, offering a flexible nonparametric alternative to dictionary-based methods such as SINDy. Through extensive numerical experiments on classic nonlinear systems, the approach demonstrates improved robustness to noise, accurate derivative estimation, and effective learning and prediction of dynamical behavior. The method provides a practical, kernel-based pipeline for denoising and dynamical-system identification without requiring hand-crafted dictionaries, with potential extensions to scalable kernel techniques for large datasets.

Abstract

Learning the governing equations from time-series data has gained increasing attention due to its potential to extract useful dynamics from real-world data. Despite significant progress, it becomes challenging in the presence of noise, especially when derivatives need to be calculated. To reduce the effect of noise, we propose a method that simultaneously fits both the derivative and trajectory from noisy time-series data. Our approach formulates derivative estimation as an inverse problem involving integral operators within the forward model, and estimates the derivative function by solving a regularization problem in a vector-valued reproducing kernel Hilbert space (vRKHS). We derive an integral-form representer theorem, which enables the computation of the regularized solution by solving a finite-dimensional problem and facilitates efficiently estimating the optimal regularization parameter. By embedding the dynamics within a vRKHS and utilizing the fitted derivative and trajectory, we can recover the underlying dynamics from noisy data by solving a linear regularization problem. Several numerical experiments are conducted to validate the effectiveness and efficiency of our method.

Paper Structure

This paper contains 14 sections, 7 theorems, 68 equations, 7 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Notations
  3. Preliminary
  4. SINDy and TV derivative estimation method
  5. Vector-valued RKHS
  6. Estimating derivative function by vRKHS regularization
  7. Practical computations for fitting derivative and trajectory simultaneously
  8. Choice of kernels
  9. Determining regularization parameter
  10. Learning dynamics from the fitted derivative and trajectory
  11. Numerical experiments
  12. Numerical comparison of the methods for estimating derivatives
  13. Numerical results for learning dynamics
  14. Conclusion

Key Result

Theorem 2.3

Suppose $K$ is an $\mathbb{R}^d$-valued reproducing kernel on $X$. Then there is a unique $\mathbb{R}^d$-valued RKHS $\mathcal{H}\subseteq (\mathbb{R}^d)^{X}$ for which $K$ is the reproducing kernel.

Figures (7)

  • Figure 1: Trajectories and noisy data. (a) Forced vibration of nonlinear pendulum, 1000 random times points in $[0,10]$ with noisy level $0.01$. (b) Lotka--Volterra equation, 2000 random times points in $[0,10]$ with noisy level $1.0$. (c) SIR model, 3000 random time points in $[0,30]$ with noisy level $5.0$. (d) Lorenz63 model, 6000 random times points in $[0,30]$ with noisy level $0.5$. (e), (f) Projections of the trajectories and noisy data for Lorenz96, 8000 random times points in $[0,30]$ with noisy level $0.1$.
  • Figure 2: Plot of L-curve for vRKHS based derivative estimation for the Lorenz63 system.
  • Figure 3: Numerical derivative of $x_1$ from noise data for Lorzen63 system with $\delta = 1$. The exact derivative is shown in blue ($--$) and the numerical derivative is shown in red (--). (a) The numerical derivative obtained by finite difference method. (b) The numerical derivative obtained by TV regularization method. (c) The numerical derivative obtained by vRKHS method.
  • Figure 4: Observation data $y_1$ with noise $\delta = 1$ and the denoised data using vRKHS. The exact data is shown in blue (--), the observation data is shown in red (-), and the denoised data using vRKHS is shown in green ($--$). (a) Exact data and denoise data for Lorenz63 on equidistributed time points. (b) Exact data and denoised data for Lotka--Volterra system on nonuniform distributed time points.
  • Figure 5: True and fitted derivatives and trajectories of Lorenz63 system by vRKHS. The noise data are taken from 4000 equistributed points in $[0,30]$ with noise level $\delta = 0.5$.
  • ...and 2 more figures

Theorems & Definitions (14)

  • Definition 2.1: Reproducing kernel
  • Definition 2.2: Vector-valued RKHS
  • Theorem 2.3: Moore-Aronszajn
  • Theorem 2.4: Representer theorem
  • Lemma 3.1
  • Proof 1
  • Theorem 3.2: Integral-form representer theorem
  • Proof 2
  • Corollary 3.3
  • Proof 3
  • ...and 4 more