Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Localisation of Regularised and Multiview Support Vector Machine Learning

Aurelian Gheondea, Cankat Tilki

TL;DR

This work generalises representer theorems to locally localised, operator-valued kernel learning for semisupervised, manifold regularised, and multiview SVMs in both finite and infinite input spaces. It shows that under mild assumptions the minimisers reside in the span of kernel evaluations, with explicit coefficient systems obtainable for differentiable losses, including a linear system for the least squares case and a nonlinear system for exponential least squares. The authors establish a framework based on bundles of Hilbert spaces (VH-spaces) and operator-valued kernels, derive representer theorems, and develop Newton-type numerical methods for the nonlinear cases, complemented by a toy example to illustrate tractability and robustness. This supports flexible modelling when outputs vary locally, views differ, or data is partly unlabeled, enabling scalable, multi-faceted SVM learning in complex spaces with potential nonconvex losses.

Abstract

We prove a few representer theorems for a localised version of the regularised and multiview support vector machine learning problem introduced by H.Q. Minh, L. Bazzani, and V. Murino, Journal of Machine Learning Research, 17(2016) 1-72, that involves operator valued positive semidefinite kernels and their reproducing kernel Hilbert spaces. The results concern general cases when convex or nonconvex loss functions and finite or infinite dimensional input spaces are considered. We show that the general framework allows infinite dimensional input spaces and nonconvex loss functions for some special cases, in particular in case the loss functions are Gateaux differentiable. Detailed calculations are provided for the exponential least squares loss functions that lead to systems of partially nonlinear equations for which a particular different types of Newton's approximation methods based on the interior point method can be used. Some numerical experiments are performed on a toy model that illustrate the tractability of the methods that we propose.

Localisation of Regularised and Multiview Support Vector Machine Learning

TL;DR

This work generalises representer theorems to locally localised, operator-valued kernel learning for semisupervised, manifold regularised, and multiview SVMs in both finite and infinite input spaces. It shows that under mild assumptions the minimisers reside in the span of kernel evaluations, with explicit coefficient systems obtainable for differentiable losses, including a linear system for the least squares case and a nonlinear system for exponential least squares. The authors establish a framework based on bundles of Hilbert spaces (VH-spaces) and operator-valued kernels, derive representer theorems, and develop Newton-type numerical methods for the nonlinear cases, complemented by a toy example to illustrate tractability and robustness. This supports flexible modelling when outputs vary locally, views differ, or data is partly unlabeled, enabling scalable, multi-faceted SVM learning in complex spaces with potential nonconvex losses.

Abstract

We prove a few representer theorems for a localised version of the regularised and multiview support vector machine learning problem introduced by H.Q. Minh, L. Bazzani, and V. Murino, Journal of Machine Learning Research, 17(2016) 1-72, that involves operator valued positive semidefinite kernels and their reproducing kernel Hilbert spaces. The results concern general cases when convex or nonconvex loss functions and finite or infinite dimensional input spaces are considered. We show that the general framework allows infinite dimensional input spaces and nonconvex loss functions for some special cases, in particular in case the loss functions are Gateaux differentiable. Detailed calculations are provided for the exponential least squares loss functions that lead to systems of partially nonlinear equations for which a particular different types of Newton's approximation methods based on the interior point method can be used. Some numerical experiments are performed on a toy model that illustrate the tractability of the methods that we propose.
Paper Structure (16 sections, 14 theorems, 171 equations, 1 algorithm)

This paper contains 16 sections, 14 theorems, 171 equations, 1 algorithm.

Table of Contents

  1. Introduction
  2. A General Semisupervised Regularised and Multiview Machine Learning Problem
  3. Operator Valued Kernels.
  4. Localisation of Semisupervised, Regularised, and Multiview Learning.
  5. A Representer Theorem.
  6. Differentiable Loss Functions
  7. Preliminary Results on Differentiable Optimisation.
  8. The Representer Theorem for Locally Differentiable Loss Functions.
  9. The Least Squares Loss Function.
  10. The Exponential Least Squares Loss Function.
  11. Available Numerical Methods for the Exponential Least Square Loss Function.
  12. A Toy Model.
  13. Proof of Theorem \ref{['t:kolmogorov']}
  14. Proof of Theorem \ref{['t:rkh']}
  15. A Direct Construction of ${\mathcal{R}}(K)$.
  16. ...and 1 more sections

Key Result

Theorem 2.1

Given an arbitrary bundle of Hilbert spaces $\boldsymbol{H}=\{{\mathcal{H}}_x\}_{x\in X}$ and an $\boldsymbol{H}$-operator valued kernel $K$, the following assertions are equivalent. In addition, if $K$ is positive semidefinite then a minimal Hilbert space linearisation $({\mathcal{K}};V)$ exists and it is unique, modulo unitary equivalence, that is, for any other minimal Hilbert space linearisat

Theorems & Definitions (32)

  • Theorem 2.1
  • Theorem 2.2
  • Remark 2.3
  • Theorem 2.4
  • Example 2.5
  • Example 2.6
  • Example 2.7
  • Example 2.8
  • Lemma 2.9
  • proof
  • ...and 22 more