An Adaptive Cost-Sensitive Learning and Recursive Denoising Framework for Imbalanced SVM Classification

TL;DR

The paper tackles imbalanced classification in SVM contexts by introducing AW-WSVM, a generalized framework that combines adaptive cost-sensitive learning with recursive denoising and a distance-based weighting scheme. It defines an Adaptive Weight Function that maps sample-to-hyperplane distances $d_i^k$ to weights $\alpha_{ki}$ via a mixed Gaussian-exponential form, emphasizing samples near the decision boundary and allowing iterative reweighting during optimization. A recursive denoising strategy filters noise by nearest-neighbor analysis to reduce minority-class outliers, and the framework is designed to be compatible with stochastic optimizers such as SGD, oBFGS, and oNAQ. Empirical results on 12 standard datasets and emotional datasets with varying imbalance ratios show consistent improvements in accuracy, G-mean, recall, and F1-score, with strong statistical support via Friedman and Nemenyi tests, highlighting its robustness and scalability for large-scale imbalanced problems.

Abstract

Category imbalance is one of the most popular and important issues in the domain of classification. Emotion classification model trained on imbalanced datasets easily leads to unreliable prediction. The traditional machine learning method tends to favor the majority class, which leads to the lack of minority class information in the model. Moreover, most existing models will produce abnormal sensitivity issues or performance degradation. We propose a robust learning algorithm based on adaptive cost-sensitivity and recursive denoising, which is a generalized framework and can be incorporated into most stochastic optimization algorithms. The proposed method uses the dynamic kernel distance optimization model between the sample and the decision boundary, which makes full use of the sample's prior information. In addition, we also put forward an effective method to filter noise, the main idea of which is to judge the noise by finding the nearest neighbors of the minority class. In order to evaluate the strength of the proposed method, we not only carry out experiments on standard datasets but also apply it to emotional classification problems with different imbalance rates (IR). Experimental results show that the proposed general framework is superior to traditional methods in Accuracy, G-mean, Recall and F1-score.

Figures (10)

• Figure 1: Decision hyperplane learned by hard-margin SVMs. Circles are support vectors.
• Figure 2: Decision hyperplane learned from imbalanced data. The solid line is the hyperplane obtained by the proposed method, while the dashed line is obtained by the standard solving algorithm.
• Figure 3: Decision hyperplane learned by hard-margin SVMs. The solid line is a decision hyperplane without noise, while the dashed line is a decision hyperplane with noise. The noises are marked as circles.
• Figure 4: AW function with different $\sigma$ ($\sigma\in\{0.5,1,1.5\}$)
• Figure 5: The process of filtering samples

Theorems & Definitions (2)

• Definition 1: AW function
• proof