Enhancing Robustness and Efficiency of Least Square Twin SVM via Granular Computing

TL;DR

This work tackles robustness and efficiency gaps in least squares twin SVM (LSTSVM) by introducing granular-ball representations (GBLSTSVM) and a large-scale variant (LS-GBLSTSVM) that incorporate structural risk minimization through regularization. The methods extend to linear and nonlinear kernels, encoding granular-ball centers and radii into the optimization framework, and employ closed-form solutions or SMO to avoid costly matrix inversions. Across 34 UCI/KEEL datasets and large NDC datasets, the proposed approaches consistently surpass baseline models in accuracy and exhibit substantial speedups (up to 1000x) while maintaining robustness to label noise. The results demonstrate improved generalization, scalability to millions of samples, and a novel dimensionality-reduction perspective via granular-ball representations.

Abstract

In the domain of machine learning, least square twin support vector machine (LSTSVM) stands out as one of the state-of-the-art models. However, LSTSVM suffers from sensitivity to noise and outliers, overlooking the SRM principle and instability in resampling. Moreover, its computational complexity and reliance on matrix inversions hinder the efficient processing of large datasets. As a remedy to the aforementioned challenges, we propose the robust granular ball LSTSVM (GBLSTSVM). GBLSTSVM is trained using granular balls instead of original data points. The core of a granular ball is found at its center, where it encapsulates all the pertinent information of the data points within the ball of specified radius. To improve scalability and efficiency, we further introduce the large-scale GBLSTSVM (LS-GBLSTSVM), which incorporates the SRM principle through regularization terms. Experiments are performed on UCI, KEEL, and NDC benchmark datasets; both the proposed GBLSTSVM and LS-GBLSTSVM models consistently outperform the baseline models.

Figures (6)

• Figure 1: Pictorial representation of the process of granular ball generation.
• Figure 2: Inseperable GBSVM model having supporting granular balls $GB_j$, supporting centers $o_j$, and supporting radii $d_j$.
• Figure 3: Granular ball least square twin support vector machine model having two non-parallel hyperplanes $f_1$ and $f_2$.
• Figure 4: Overall architectural framework of the proposed linear GBLSTSVM model.
• Figure 5: The effect of hyperparameter $(c_1, c_2)$ tuning on the accuracy (ACC) of some UCI and KEEL datasets on the performance of linear GBLSTSVM.