Tverberg's theorem and multi-class support vector machines

TL;DR

This work establishes a bridge between Tverberg-type geometric results and multi-class SVMs by leveraging Sarkaria's linear-algebraic construction to embed a $k$-class problem into a higher-dimensional space. The authors define two multi-class SVM variants, Simple TSVM and TSVM, which require the weaker condition $\bigcap_{i=1}^k \mathrm{conv}(A_i)=\emptyset$ rather than pairwise separability, and show how standard SVM machinery applies in the elevated space $\mathbb{R}^{(d+1)(k-1)}$. They provide a concrete geometric characterization of the corresponding support vectors, a rigorous mapping back to $k$ half-spaces in $\mathbb{R}^d$, and algorithmic implications including a randomized, expected linear-time approach for fixed $k$ and $d$. The framework also proves important invariance and equivariance properties under orthogonal transformations and translations, underscoring the theoretical robustness and potential practical impact for multi-class classification under relaxed separability assumptions.

Abstract

We show how, using linear-algebraic tools developed to prove Tverberg's theorem in combinatorial geometry, we can design new models of multi-class support vector machines (SVMs). These supervised learning protocols require fewer conditions to classify sets of points, and can be computed using existing binary SVM algorithms in higher-dimensional spaces, including soft-margin SVM algorithms. We describe how the theoretical guarantees of standard support vector machines transfer to these new classes of multi-class support vector machines. We give a new simple proof of a geometric characterization of support vectors for largest margin SVMs by Veelaert.

Figures (3)

• Figure 1: (1) An example of an SVM, we emphasize the support hyperplanes parallel to the generated hyperplane for each class. (2) An example of an multi-class SVM under the proposed model. Notice that it is not possible to separate any two classes of points with a hyperplane. (3) The half-spaces in part (2) can be used to classify space using convex regions. The model can distinguish regions where it is ambiguous.
• Figure 2: An example of a largest-margin SVM with two sets. If we project the support vectors onto the separating hyperplane, the convex hulls of the projections of different sides intersect.
• Figure 3: This figure shows the process to find (TSVM) for two sets of points. First we embed the sets in $U_1$, then we reflect $A_2$ across the origin to obtain their representatives in $U_2$. We take the convex hull of $Y_1$ and $Y_2$ and intersect it with $R$, which in the figure gives us a hexagon. We take the closest point $p$ to the origin in $\mathop{\mathrm{conv}}\nolimits(Y)\cap R$ and construct a hyperplane parallel to the facet containing $p$ of $\mathop{\mathrm{conv}}\nolimits(Y)$ through the origin. This hyperplane intersects $U_1$ in the largest-margin SVM for the original sets.

Theorems & Definitions (24)

• Theorem 2.0.1
• proof
• Lemma 3.0.1
• proof
• Lemma 3.0.2
• proof
• Lemma 3.0.3
• proof
• Lemma 3.0.4
• proof