Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Kernel Sum of Squares for Data Adapted Kernel Learning of Dynamical Systems from Data: A global optimization approach

Daniel Lengyel, Panos Parpas, Boumediene Hamzi, Houman Owhadi

TL;DR

The paper tackles the challenge of learning kernels for data-driven dynamical system emulation by addressing the nonconvex kernel-parameter optimization problem with a global KSOS framework. By replacing local gradient searches with a kernel-based surrogate and a finite SOS-like constraint set, the approach yields data-adapted kernels that better capture underlying dynamics, as demonstrated on the Logistic map, Henon map, and Lorenz system. Across these experiments, KSOS consistently improves the relative-$\rho$ metric and often achieves lower test mean errors and more stable trajectory predictions than gradient descent, highlighting the method's potential for robust time-series forecasting in chaotic regimes. The work also discusses practical aspects, such as sampling strategies, solver choices, and the trade-offs between global optimality and computational cost, suggesting a productive path toward hybrid optimization schemes and broader applicability in time-series analysis of complex systems.

Abstract

This paper examines the application of the Kernel Sum of Squares (KSOS) method for enhancing kernel learning from data, particularly in the context of dynamical systems. Traditional kernel-based methods, despite their theoretical soundness and numerical efficiency, frequently struggle with selecting optimal base kernels and parameter tuning, especially with gradient-based methods prone to local optima. KSOS mitigates these issues by leveraging a global optimization framework with kernel-based surrogate functions, thereby achieving more reliable and precise learning of dynamical systems. Through comprehensive numerical experiments on the Logistic Map, Henon Map, and Lorentz System, KSOS is shown to consistently outperform gradient descent in minimizing the relative-$ρ$ metric and improving kernel accuracy. These results highlight KSOS's effectiveness in predicting the behavior of chaotic dynamical systems, demonstrating its capability to adapt kernels to underlying dynamics and enhance the robustness and predictive power of kernel-based approaches, making it a valuable asset for time series analysis in various scientific fields.

Kernel Sum of Squares for Data Adapted Kernel Learning of Dynamical Systems from Data: A global optimization approach

TL;DR

The paper tackles the challenge of learning kernels for data-driven dynamical system emulation by addressing the nonconvex kernel-parameter optimization problem with a global KSOS framework. By replacing local gradient searches with a kernel-based surrogate and a finite SOS-like constraint set, the approach yields data-adapted kernels that better capture underlying dynamics, as demonstrated on the Logistic map, Henon map, and Lorenz system. Across these experiments, KSOS consistently improves the relative- metric and often achieves lower test mean errors and more stable trajectory predictions than gradient descent, highlighting the method's potential for robust time-series forecasting in chaotic regimes. The work also discusses practical aspects, such as sampling strategies, solver choices, and the trade-offs between global optimality and computational cost, suggesting a productive path toward hybrid optimization schemes and broader applicability in time-series analysis of complex systems.

Abstract

This paper examines the application of the Kernel Sum of Squares (KSOS) method for enhancing kernel learning from data, particularly in the context of dynamical systems. Traditional kernel-based methods, despite their theoretical soundness and numerical efficiency, frequently struggle with selecting optimal base kernels and parameter tuning, especially with gradient-based methods prone to local optima. KSOS mitigates these issues by leveraging a global optimization framework with kernel-based surrogate functions, thereby achieving more reliable and precise learning of dynamical systems. Through comprehensive numerical experiments on the Logistic Map, Henon Map, and Lorentz System, KSOS is shown to consistently outperform gradient descent in minimizing the relative- metric and improving kernel accuracy. These results highlight KSOS's effectiveness in predicting the behavior of chaotic dynamical systems, demonstrating its capability to adapt kernels to underlying dynamics and enhance the robustness and predictive power of kernel-based approaches, making it a valuable asset for time series analysis in various scientific fields.
Paper Structure (21 sections, 3 theorems, 23 equations, 36 figures, 9 tables)

This paper contains 21 sections, 3 theorems, 23 equations, 36 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Statement of the Problem
  3. On Kernel Learning
  4. Gradient-based methods
  5. Kernel Sum of Squares as a Global Method
  6. Numerical Experiments
  7. Algorithmic Setup
  8. Gradient Descent
  9. Kernel Sum of Squares
  10. Evaluation
  11. Logistic Map
  12. The Hénon Map
  13. Lorentz System
  14. Discussion
  15. Conclusion
  16. ...and 6 more sections

Key Result

Proposition A.1

If $k$ is a reproducing kernel of a Hilbert space ${\mathcal{H}}$, then

Figures (36)

  • Figure 1: Rho Relative on Logistic Map Training Set.
  • Figure 2: KSOS Test Trajectory
  • Figure 3: Gradient Descent Test Trajectory
  • Figure 4: KSOS Train Trajectory
  • Figure 5: Gradient Descent Train Trajectory
  • ...and 31 more figures

Theorems & Definitions (4)

  • Definition A.1
  • Proposition A.1
  • Theorem A.1
  • Theorem A.2