Kernel-Free Universum Quadratic Surface Twin Support Vector Machines for Imbalanced Data
Hossein Moosaei, Milan Hladík, Ahmad Mousavi, Zheming Gao, Haojie Fu
TL;DR
This work addresses binary classification with severe class imbalance by developing kernel-free, Universum-augmented quadratic twin SVMs. The authors introduce both hinge-loss and least-squares formulations, including LS-$ rak{U}$-QTSVM and Im-$ rak{U}$-QTSVM, to model nonlinear boundaries with minimal computational overhead. They provide theoretical guarantees on existence, uniqueness, and complexity, and demonstrate strong empirical performance on artificial and public benchmark datasets, with statistical tests confirming superiority over existing methods. The methods enable effective minority-class support through Universum points, undersampling, and Hessian regularization, offering scalable and generalizable solutions for imbalanced classification tasks.
Abstract
Binary classification tasks with imbalanced classes pose significant challenges in machine learning. Traditional classifiers often struggle to accurately capture the characteristics of the minority class, resulting in biased models with subpar predictive performance. In this paper, we introduce a novel approach to tackle this issue by leveraging Universum points to support the minority class within quadratic twin support vector machine models. Unlike traditional classifiers, our models utilize quadratic surfaces instead of hyperplanes for binary classification, providing greater flexibility in modeling complex decision boundaries. By incorporating Universum points, our approach enhances classification accuracy and generalization performance on imbalanced datasets. We generated four artificial datasets to demonstrate the flexibility of the proposed methods. Additionally, we validated the effectiveness of our approach through empirical evaluations on benchmark datasets, showing superior performance compared to conventional classifiers and existing methods for imbalanced classification.