Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Kernel Estimation for Nonlinear Dynamics

Marie-Christine Düker, Adam Waterbury

TL;DR

The paper develops nonasymptotic guarantees for kernel ridge regression estimators of nonlinear vector autoregressions in the presence of temporal and cross-sectional dependence by leveraging RKHS representations with multivariate kernels. It models the dynamics as an $(\mathcal{X}^p)$-valued Markov chain and derives high-probability bounds for the estimator error $||\hat{g}_T - g||_\infty$ through concentration results on quadratic forms of dependent data, accommodating both finite and infinite Mercer representations. A key technical contribution is a Hoeffding-type inequality for such quadratic forms, alongside conditions on multivariate kernels that guarantee optimal convergence rates up to logarithmic factors. The framework unifies stochastic regression and nonlinear VAR settings, applies to Gaussian and non-Gaussian noise, and provides concrete kernel examples (Gaussian, polynomial, Mercer sigmoid) where the results hold. Overall, the work extends kernel-based nonparametric regression to dependent time-series with rich nonlinear dynamics and yields explicit, rate-optimal convergence guarantees under broad kernel assumptions.

Abstract

Many scientific problems involve data exhibiting both temporal and cross-sectional dependencies. While linear dependencies have been extensively studied, the theoretical analysis of regression estimators under nonlinear dependencies remains scarce. This work studies a kernel-based estimation procedure for nonlinear dynamics within the reproducing kernel Hilbert space framework, focusing on nonlinear vector autoregressive models. We derive nonasymptotic probabilistic bounds on the deviation between a regularized kernel estimator and the nonlinear regression function. A key technical contribution is a concentration bound for quadratic forms of stochastic matrices in the presence of dependent data, which is of independent interest. Additionally, we characterize conditions on multivariate kernels that guarantee optimal convergence rates.

Kernel Estimation for Nonlinear Dynamics

TL;DR

The paper develops nonasymptotic guarantees for kernel ridge regression estimators of nonlinear vector autoregressions in the presence of temporal and cross-sectional dependence by leveraging RKHS representations with multivariate kernels. It models the dynamics as an -valued Markov chain and derives high-probability bounds for the estimator error through concentration results on quadratic forms of dependent data, accommodating both finite and infinite Mercer representations. A key technical contribution is a Hoeffding-type inequality for such quadratic forms, alongside conditions on multivariate kernels that guarantee optimal convergence rates up to logarithmic factors. The framework unifies stochastic regression and nonlinear VAR settings, applies to Gaussian and non-Gaussian noise, and provides concrete kernel examples (Gaussian, polynomial, Mercer sigmoid) where the results hold. Overall, the work extends kernel-based nonparametric regression to dependent time-series with rich nonlinear dynamics and yields explicit, rate-optimal convergence guarantees under broad kernel assumptions.

Abstract

Many scientific problems involve data exhibiting both temporal and cross-sectional dependencies. While linear dependencies have been extensively studied, the theoretical analysis of regression estimators under nonlinear dependencies remains scarce. This work studies a kernel-based estimation procedure for nonlinear dynamics within the reproducing kernel Hilbert space framework, focusing on nonlinear vector autoregressive models. We derive nonasymptotic probabilistic bounds on the deviation between a regularized kernel estimator and the nonlinear regression function. A key technical contribution is a concentration bound for quadratic forms of stochastic matrices in the presence of dependent data, which is of independent interest. Additionally, we characterize conditions on multivariate kernels that guarantee optimal convergence rates.

Paper Structure

This paper contains 18 sections, 24 theorems, 188 equations.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Kernel ridge regression
  4. Assumptions
  5. Concentration results for kernel functions
  6. Main results
  7. Examples
  8. Infinite Mercer representation
  9. Finite Mercer representation
  10. Stochastic regression model
  11. Auxiliary results
  12. Reproducing kernel Hilbert spaces
  13. Geometric ergodicity
  14. Abstract estimates
  15. Proofs of main results
  16. ...and 3 more sections

Key Result

Theorem 3.1

Suppose Assumptions ass:gbounded, ass:kernelbound--ass:momentboundtail, and ass:subgaussiannoise. There are constants $c_1,c_2 \in (0,\infty)$ and $L, \operatorname{c_0} \in (0,\infty)$ such that if and then, with probability at least $1- c_1 T^{-c_2}$, If Assumption ass:separable_kernel_multi is satisfied with $M = \infty$, then we take $L \doteq M(T)$, where $M(T)$ is as in Assumption ass:mo

Theorems & Definitions (48)

  • Remark 2.1
  • Remark 2.2
  • Remark 2.3
  • Theorem 3.1
  • Theorem 3.2
  • Theorem 3.3
  • Theorem 3.4
  • proof
  • Theorem 3.5
  • proof
  • ...and 38 more