Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust support vector model based on bounded asymmetric elastic net loss for binary classification

Haiyan Du, Hu Yang

TL;DR

By proving the violation tolerance upper bound (VTUB) of BAEN-SVM, it is shown that the model is geometrically well-defined and derived that the influence function of BAEN-SVM is bounded, providing a theoretical guarantee of its robustness to noise.

Abstract

In this paper, we propose a novel bounded asymmetric elastic net ($L_{baen}$) loss function and combine it with the support vector machine (SVM), resulting in the BAEN-SVM. The $L_{baen}$ is bounded and asymmetric and can degrade to the asymmetric elastic net hinge loss, pinball loss, and asymmetric least squares loss. BAEN-SVM not only effectively handles noise-contaminated data but also addresses the geometric irrationalities in the traditional SVM. By proving the violation tolerance upper bound (VTUB) of BAEN-SVM, we show that the model is geometrically well-defined. Furthermore, we derive that the influence function of BAEN-SVM is bounded, providing a theoretical guarantee of its robustness to noise. The Fisher consistency of the model further ensures its generalization capability. Since the \( L_{\text{baen}} \) loss is non-convex, we designed a clipping dual coordinate descent-based half-quadratic algorithm to solve the non-convex optimization problem efficiently. Experimental results on artificial and benchmark datasets indicate that the proposed method outperforms classical and advanced SVMs, particularly in noisy environments.

Robust support vector model based on bounded asymmetric elastic net loss for binary classification

TL;DR

By proving the violation tolerance upper bound (VTUB) of BAEN-SVM, it is shown that the model is geometrically well-defined and derived that the influence function of BAEN-SVM is bounded, providing a theoretical guarantee of its robustness to noise.

Abstract

In this paper, we propose a novel bounded asymmetric elastic net () loss function and combine it with the support vector machine (SVM), resulting in the BAEN-SVM. The is bounded and asymmetric and can degrade to the asymmetric elastic net hinge loss, pinball loss, and asymmetric least squares loss. BAEN-SVM not only effectively handles noise-contaminated data but also addresses the geometric irrationalities in the traditional SVM. By proving the violation tolerance upper bound (VTUB) of BAEN-SVM, we show that the model is geometrically well-defined. Furthermore, we derive that the influence function of BAEN-SVM is bounded, providing a theoretical guarantee of its robustness to noise. The Fisher consistency of the model further ensures its generalization capability. Since the loss is non-convex, we designed a clipping dual coordinate descent-based half-quadratic algorithm to solve the non-convex optimization problem efficiently. Experimental results on artificial and benchmark datasets indicate that the proposed method outperforms classical and advanced SVMs, particularly in noisy environments.
Paper Structure (21 sections, 5 theorems, 84 equations, 7 figures, 6 tables, 1 algorithm)

This paper contains 21 sections, 5 theorems, 84 equations, 7 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Elastic net loss for SVM
  4. Bounded loss functions for SVM
  5. Bounded Asymmetric Elastic Net Loss-Based SVM
  6. The BAEN-SVM Model
  7. The clipDCD-based HQ Algorithm for BAEN-SVM
  8. Relationships with other models
  9. Properties of BAEN-SVM
  10. Geometrical Rationality
  11. Fisher Consistency
  12. Noise Insensitivity
  13. Robust to Label Noise
  14. Robust to Feature noise
  15. Complexity Analysis
  16. ...and 6 more sections

Key Result

Theorem 1

Given $n$ samples and parameter $p\in (0,1)$, if the training samples $x_i$ and $x_j$ belong to the same class and both violate the constraints of BAEN-SVM, for $\xi_{i}$ and $\xi_{j}$ are estimated by BAEN-SVM, we have where $\tilde{X}=(X,e)$, $\left\|\tilde{X}\right\|_{F}$ is the Frobenius norm of $\tilde{X}$, $\vartheta_{1}$ and $\vartheta_{n}$ are the smallest and the largest eigenvalues of $

Figures (7)

  • Figure 1: The relationship between sample $x_i$ and slack variable $\xi_i$ for SVM
  • Figure 2: Different parameter of $L_{{baen}}$
  • Figure 3: Loss functions with different parameters
  • Figure 4: Linear separating hyperplanes(black solid lines of Hing-SVM,Pin-SVM,LS-SVM,ALS-SVM,EN-SVM,BAEN-SVM. The green solid line is the Bayes classifier.
  • Figure 5: Linear separating hyperplanes(black solid lines of Hing-SVM,Pin-SVM,LS-SVM,ALS-SVM,EN-SVM,BAEN-SVM. The green solid line is the Bayes classifier.
  • ...and 2 more figures

Theorems & Definitions (11)

  • Theorem 1
  • proof
  • Remark 1
  • Remark 2
  • Remark 3
  • Theorem 2
  • Lemma 1
  • Theorem 3
  • proof
  • Theorem 4
  • ...and 1 more