Vector-Valued Reproducing Kernel Banach Spaces for Neural Networks and Operators

TL;DR

This work develops a general framework of vector-valued reproducing kernel Banach spaces (vv-RKBS) to unify neural networks and neural operators under a kernel-based, Banach-space lens. By introducing adjoint vv-RKBS pairs and integral/neural vv-RKBS constructions, the authors relax classical assumptions (e.g., reflexivity, separability) and provide a Representer Theorem that yields sparse, kernel-based representations for learning in these spaces. They demonstrate that $\mathbb{R}^d$-valued nets, DeepONets, and Hypernetworks naturally live inside integral and neural vv-RKBS, with explicit kernel structures and reproducing properties guiding optimization. The resulting theory bridges function-space perspectives of neural architectures with kernel methods, enabling principled analysis and potential pathway to kernel chaining for deep networks and operator learning.

Abstract

Recently, there has been growing interest in characterizing the function spaces underlying neural networks. While shallow and deep scalar-valued neural networks have been linked to scalar-valued reproducing kernel Banach spaces (RKBS), $\mathbb{R}^d$-valued neural networks and neural operator models remain less understood in the RKBS setting. To address this gap, we develop a general definition of vector-valued RKBS (vv-RKBS), which inherently includes the associated reproducing kernel. Our construction extends existing definitions by avoiding restrictive assumptions such as symmetric kernel domains, finite-dimensional output spaces, reflexivity, or separability, while still recovering familiar properties of vector-valued reproducing kernel Hilbert spaces (vv-RKHS). We then show that shallow $\mathbb{R}^d$-valued neural networks are elements of a specific vv-RKBS, namely an instance of the integral and neural vv-RKBS. To also explore the functional structure of neural operators, we analyze the DeepONet and Hypernetwork architectures and demonstrate that they too belong to an integral and neural vv-RKBS. In all cases, we establish a Representer Theorem, showing that optimization over these function spaces recovers the corresponding neural architectures.

Figures (1)

• Figure 1: From vv-RKHS to adjoint pair of vv-RKBS. In a vv-RKHS, functions $f \colon X \to \mathcal{U}$ take values in a Hilbert space $\mathcal{U}$ and admit a reproducing kernel $K \colon X \times X \to \mathcal{L}(\mathcal{U})$. In the vv-RKBS setting, we instead use Banach spaces $\mathcal{B}$ and $\mathcal{B}^\diamond$ for functions mapping to a dual pair $(\mathcal{U}, \mathcal{U}^\diamond)$, and replace inner products with duality pairings. This breaks the symmetry in the domain, replacing $X \times X$ with $X \times \Omega$, and requires twin operators in place of $\mathcal{L}(\mathcal{U})$ to accommodate the asymmetry.

Theorems & Definitions (62)

• definition 2.1: Vector-valued Reproducing Kernel Hilbert space
• definition 2.2: Kernel definition vv-RKHS
• theorem 2.1
• proof
• definition 2.3: Feature map definition vv-RKHS
• definition 3.1: Vector-valued Reproducing Kernel Banach space
• definition 3.2: Feature map definition vv-RKBS
• remark 3.1
• definition 3.3: Dual pair fabian2011banach
• definition 3.4: Adjoint pair of scalar RKBS