Margin-aware Fuzzy Rough Feature Selection: Bridging Uncertainty Characterization and Pattern Classification

TL;DR

MAFRFS addresses a key gap in fuzzy rough feature selection by linking uncertainty reduction with discriminative label-class margins. It introduces within-class and between-class margins and their ratio (WBMR) to steer forward greedy feature selection toward more separable class structures, while integrating multiple uncertainty measures. Empirical results on 15 datasets show consistent performance gains over FRFS and six state-of-the-art baselines across CART, SVM, and KNN, with FE+ and FJE+ often achieving the largest improvements. The framework is scalable, adaptable to various uncertainty measures, and enhances classifier generalization by balancing compactness and separation in the reduced feature space.

Abstract

Fuzzy rough feature selection (FRFS) is an effective means of addressing the curse of dimensionality in high-dimensional data. By removing redundant and irrelevant features, FRFS helps mitigate classifier overfitting, enhance generalization performance, and lessen computational overhead. However, most existing FRFS algorithms primarily focus on reducing uncertainty in pattern classification, neglecting that lower uncertainty does not necessarily result in improved classification performance, despite it commonly being regarded as a key indicator of feature selection effectiveness in the FRFS literature. To bridge uncertainty characterization and pattern classification, we propose a Margin-aware Fuzzy Rough Feature Selection (MAFRFS) framework that considers both the compactness and separation of label classes. MAFRFS effectively reduces uncertainty in pattern classification tasks, while guiding the feature selection towards more separable and discriminative label class structures. Extensive experiments on 15 public datasets demonstrate that MAFRFS is highly scalable and more effective than FRFS. The algorithms developed using MAFRFS outperform six state-of-the-art feature selection algorithms.

Figures (6)

• Figure 1: Overall FRFS framework and underlying processing
• Figure 2: An illustration of within-class margin
• Figure 3: An illustration of between-class margin
• Figure 4: Overall MAFRFS framework and underlying processing
• Figure 5: Predictive performance (CART) using the top 30%, 50%, 70%, and 90% of ranked features across $15$ datasets.