Featured Reproducing Kernel Banach Spaces for Learning and Neural Networks
Isabel de la Higuera, Francisco Herrera, M. Victoria Velasco
TL;DR
This work extends kernel-based learning beyond reproducing kernel Hilbert spaces by introducing featured reproducing kernel Banach spaces (featured RKBS). It identifies precise structural conditions—centered on feature maps and predual representations—under which Banach-space learning problems admit kernels and finite representer-type solutions, and it shows how fixed-architecture neural networks instantiate vector-valued featured RKBSs. The authors develop norm interpolation and regularization results, establish conditional representer theorems, and extend the framework to vector-valued settings, enabling multi-output learning and neural-network interpretations. The framework provides a unified functional-analytic lens for kernel methods and neural networks, clarifying when kernel-based principles extend beyond Hilbert spaces and outlining foundational avenues for future exploration in optimization dynamics and generalization.
Abstract
Reproducing kernel Hilbert spaces provide a foundational framework for kernel-based learning, where regularization and interpolation problems admit finite-dimensional solutions through classical representer theorems. Many modern learning models, however -- including fixed-architecture neural networks equipped with non-quadratic norms -- naturally give rise to non-Hilbertian geometries that fall outside this setting. In Banach spaces, continuity of point-evaluation functionals alone is insufficient to guarantee feature representations or kernel-based learning formulations. In this work, we develop a functional-analytic framework for learning in Banach spaces based on the notion of featured reproducing kernel Banach spaces. We identify the precise structural conditions under which feature maps, kernel constructions, and representer-type results can be recovered beyond the Hilbertian regime. Within this framework, supervised learning is formulated as a minimal-norm interpolation or regularization problem, and existence results together with conditional representer theorems are established. We further extend the theory to vector-valued featured reproducing kernel Banach spaces and show that fixed-architecture neural networks naturally induce special instances of such spaces. This provides a unified function-space perspective on kernel methods and neural networks and clarifies when kernel-based learning principles extend beyond reproducing kernel Hilbert spaces.