Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Random Features for Operator-Valued Kernels: Bridging Kernel Methods and Neural Operators

Mike Nguyen, Nicole Mücke

TL;DR

This work establishes minimax rates in the well-specified case and also in the misspecified case, where the target is not contained in the reproducing kernel Hilbert space, and further generalizes the setting to operator-valued kernels.

Abstract

In this work, we investigate the generalization properties of random feature methods. Our analysis extends prior results for Tikhonov regularization to a broad class of spectral regularization techniques and further generalizes the setting to operator-valued kernels. This unified framework enables a rigorous theoretical analysis of neural operators and neural networks through the lens of the Neural Tangent Kernel (NTK). In particular, it allows us to establish optimal learning rates and provides a good understanding of how many neurons are required to achieve a given accuracy. Furthermore, we establish minimax rates in the well-specified case and also in the misspecified case, where the target is not contained in the reproducing kernel Hilbert space. These results sharpen and complete earlier findings for specific kernel algorithms.

Random Features for Operator-Valued Kernels: Bridging Kernel Methods and Neural Operators

TL;DR

This work establishes minimax rates in the well-specified case and also in the misspecified case, where the target is not contained in the reproducing kernel Hilbert space, and further generalizes the setting to operator-valued kernels.

Abstract

In this work, we investigate the generalization properties of random feature methods. Our analysis extends prior results for Tikhonov regularization to a broad class of spectral regularization techniques and further generalizes the setting to operator-valued kernels. This unified framework enables a rigorous theoretical analysis of neural operators and neural networks through the lens of the Neural Tangent Kernel (NTK). In particular, it allows us to establish optimal learning rates and provides a good understanding of how many neurons are required to achieve a given accuracy. Furthermore, we establish minimax rates in the well-specified case and also in the misspecified case, where the target is not contained in the reproducing kernel Hilbert space. These results sharpen and complete earlier findings for specific kernel algorithms.
Paper Structure (6 sections, 1 theorem, 24 equations, 1 table)

This paper contains 6 sections, 1 theorem, 24 equations, 1 table.

Table of Contents

  1. Introduction
  2. Mathematical Framework
  3. Motivation: Generalization Bounds for Neural Operators
  4. Kernel Methods, Random Features, and Regularization: General Setup
  5. Main Results
  6. Assumptions and Main Results

Key Result

Theorem 3.4

Suppose Assumptions ass:input--ass:dim hold. Let $\{\phi_\lambda\}_\lambda$ be a family of regularization functions with qualification $\nu >0$. Let $\delta \in (0,1)$ and choose

Theorems & Definitions (2)

  • Definition 2.2: Regularization function
  • Theorem 3.4