Analysis of Regularized Learning in Banach Spaces for Linear-functional Data
Qi Ye
TL;DR
The paper develops a rigorous theory of regularized learning in Banach spaces for linear-functional data, addressing when exact risks are unknown by leveraging regularized empirical risks $R_n(f) + \lambda \Phi(\|f\|)$ and the weak* topology induced by a predual. It proves generalized representer and pseudo-approximation theorems that reduce infinite-dimensional problems to finite-dimensional optimizations, and establishes convergence theorems ensuring that approximate solutions converge to the true minimizer under verifiable conditions on data and losses. The framework accommodates multi-loss functions, composite algorithms, and approximate models, enabling connections to SVMs, neural networks, proximal methods, and Douglas–Rachford schemes within a unified theoretical setting. The results provide guidance for choosing adaptive regularization and for constructing computable, well-posed estimators in ill-posed scenarios, with broad applicability to inverse problems, multiscale modeling, and multimodal data integration. Overall, the work offers a principled pathway to globally approximate unknown problems through locally informative linear-functional data, under robust weak* convergence guarantees.
Abstract
This article delves into the study of the theory of regularized learning in Banach spaces for linear-functional data. It encompasses discussions on representer theorems, pseudo-approximation theorems, and convergence theorems. Regularized learning is designed to minimize regularized empirical risks over a Banach space. The empirical risks are calculated by utilizing training data and multi-loss functions. The input training data are composed of linear functionals in a predual space of the Banach space to capture discrete local information from multimodal data and multiscale models. Through the regularized learning, approximations of the exact solution to an unidentified or uncertain original problem are globally achieved. In the convergence theorems, the convergence of the approximate solutions to the exact solution is established through the utilization of the weak* topology of the Banach space. The theorems of regularized learning are utilized in the interpretation of classical machine learning, such as support vector machines and artificial neural networks.