GL-TSVM: A robust and smooth twin support vector machine with guardian loss function

TL;DR

This work introduces the Guardian loss (G-loss), an asymmetric, bounded, and smooth loss function, to robustify twin SVMs. By integrating the G-loss with SRM-regularized TSVM, the authors develop GL-TSVM with both linear and kernelized variants and present iterative optimization schemes, including efficient Sherman–Morrison–Woodbury-based updates. Comprehensive experiments on 25 UCI/KEEL datasets and biomedical datasets (BreaKHis, schizophrenia) show GL-TSVM achieving state-of-the-art or highly competitive accuracy, illustrating improved robustness to noise and outliers. While the approach incurs matrix inversions that limit large-scale applicability, the results underscore GL-TSVM’s potential for robust classification in diverse domains and motivate future work on inversion-free formulations and broader integrations of the G-loss concept.

Abstract

Twin support vector machine (TSVM), a variant of support vector machine (SVM), has garnered significant attention due to its $3/4$ times lower computational complexity compared to SVM. However, due to the utilization of the hinge loss function, TSVM is sensitive to outliers or noise. To remedy it, we introduce the guardian loss (G-loss), a novel loss function distinguished by its asymmetric, bounded, and smooth characteristics. We then fuse the proposed G-loss function into the TSVM and yield a robust and smooth classifier termed GL-TSVM. Further, to adhere to the structural risk minimization (SRM) principle and reduce overfitting, we incorporate a regularization term into the objective function of GL-TSVM. To address the optimization challenges of GL-TSVM, we devise an efficient iterative algorithm. The experimental analysis on UCI and KEEL datasets substantiates the effectiveness of the proposed GL-TSVM in comparison to the baseline models. Moreover, to showcase the efficacy of the proposed GL-TSVM in the biomedical domain, we evaluated it on the breast cancer (BreaKHis) and schizophrenia datasets. The outcomes strongly demonstrate the competitiveness of the proposed GL-TSVM against the baseline models.

Figures (1)

• Figure 1: Visual illustration of baseline and proposed G-loss function. (a) Pinball loss function with $\tau=0$, $\tau=0.2$, and $\tau=0.5$. (b) Huber loss function with $\theta=0.5$ and $\theta=1$. (c) Correntropy-induced loss function with $\rho=0.5$, $\rho=1$, and $\rho=1.5$. (d) LINEX loss function with $a=0.5$, $a=1$, and $a=1.5$. (e) Proposed G-loss function with $a=0.5$, $a=1$, $a=1.5$, and $a=2$.