Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Decomposition of one-layer neural networks via the infinite sum of reproducing kernel Banach spaces

Seungcheol Shin, Myungjoo Kang

TL;DR

This work addresses the structural understanding of integral Reproducing Kernel Banach Spaces (RKBS) and the hypothesis spaces of infinitely wide neural networks. It introduces an infinite sum of RKBSs and proves its compatibility with the direct sum of their feature spaces, enabling a decomposition of the integral RKBS $\mathcal{F}_{\sigma}(\mathcal{X},\Omega)$ into a sum of $p$-norm RKBSs $\mathcal{L}_{\sigma}^{1}(\mu_i)$ via maximal singular families. The main contributions are (i) a rigorous construction and characterization of the sum of RKBSs and its isometric relationship to the direct-sum feature-space representation, (ii) a decomposition of the integral RKBS into separable components that clarifies inclusion relations and algebraic/topological properties, and (iii) applications to existence of general solutions for one-layer networks, reformulation schemes that move between feature- and hypothesis-spaces, and a revisited representer theorem in the RKBS framework. The results provide a structural lens for learning in RKBS spaces and suggest practical benefits for designing modular, kernel-based learning systems and potential advances in multiple kernel learning within RKBS settings.

Abstract

In this paper, we define the sum of RKBSs using the characterization theorem of RKBSs and show that the sum of RKBSs is compatible with the direct sum of feature spaces. Moreover, we decompose the integral RKBS into the sum of $p$-norm RKBSs. Finally, we provide applications for the structural understanding of the integral RKBS class.

Decomposition of one-layer neural networks via the infinite sum of reproducing kernel Banach spaces

TL;DR

This work addresses the structural understanding of integral Reproducing Kernel Banach Spaces (RKBS) and the hypothesis spaces of infinitely wide neural networks. It introduces an infinite sum of RKBSs and proves its compatibility with the direct sum of their feature spaces, enabling a decomposition of the integral RKBS into a sum of -norm RKBSs via maximal singular families. The main contributions are (i) a rigorous construction and characterization of the sum of RKBSs and its isometric relationship to the direct-sum feature-space representation, (ii) a decomposition of the integral RKBS into separable components that clarifies inclusion relations and algebraic/topological properties, and (iii) applications to existence of general solutions for one-layer networks, reformulation schemes that move between feature- and hypothesis-spaces, and a revisited representer theorem in the RKBS framework. The results provide a structural lens for learning in RKBS spaces and suggest practical benefits for designing modular, kernel-based learning systems and potential advances in multiple kernel learning within RKBS settings.

Abstract

In this paper, we define the sum of RKBSs using the characterization theorem of RKBSs and show that the sum of RKBSs is compatible with the direct sum of feature spaces. Moreover, we decompose the integral RKBS into the sum of -norm RKBSs. Finally, we provide applications for the structural understanding of the integral RKBS class.
Paper Structure (21 sections, 15 theorems, 77 equations, 1 figure)

This paper contains 21 sections, 15 theorems, 77 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Related Work
  3. Organization
  4. Preliminaries and notations
  5. Direct sum of normed vector spaces
  6. Review of measure theory
  7. Reproducing kernel Banach spaces
  8. Definition of RKBS
  9. Characterization of RKBSs
  10. One-layer neural networks
  11. Integral RKBS and p-norm RKBS
  12. Infinite sum of reproducing kernel Banach spaces
  13. Main Results
  14. Compatibility between the sum of RKBSs and the direct sum of feature spaces
  15. Decomposition of one-layer neural networks
  16. ...and 6 more sections

Key Result

Theorem 3.3

A linear subspace $\mathcal{B}$ of $\mathbb{R}^{\mathcal{X}}$ is an RKBS on $\mathcal{X}$ if and only if there exists a Banach space $\Psi$ and a map $\psi:\mathcal{X} \rightarrow \Psi^{*}$ such that $\mathcal{B} = \operatorname{im}(A) = \{f: \exists \nu \in \Psi \text{ s.t. } A(\nu) = f\}$ with the

Figures (1)

  • Figure 1: Commutative diagram for the compatibility

Theorems & Definitions (38)

  • Definition 2.1: The direct sum of normed vector spaces conway1997course
  • Definition 3.1
  • Definition 3.2: Definition of reproducing kernel Banach space bartolucci2023understandinglin2022reproducing
  • Theorem 3.3: Characterization of RKBSs bartolucci2023understandingcombettes2018regularized
  • Definition 3.4: A class of integral RKBSs, associated with the function $\sigma$ bartolucci2023understandingspek2022duality
  • Proposition 3.5
  • proof
  • Definition 3.6: A class of p-Norm RKBS, associated with the function $\sigma$ spek2022duality
  • Proposition 3.7: Infinite sum of reproducing kernel Banach spaces
  • proof
  • ...and 28 more