S$^2$FS: Spatially-Aware Separability-Driven Feature Selection in Fuzzy Decision Systems

TL;DR

This work tackles the challenge of feature selection for fuzzy decision systems by addressing the mismatch between conventional evaluation criteria and learning performance, and by incorporating spatial directional information into class separability. It introduces a spatially-aware separability criterion that unifies within-class compactness and between-class separation through distance and directional components, formalized as $\text{Sep}^{F'}_L = \frac{\Lambda^{F',L}_{dis} + \beta\Lambda^{F',L}_{dir}}{\Theta^{F',L}_{dis} + \alpha\Theta^{F',L}_{dir}}$, and applies a forward greedy algorithm (S^2FS) to select informative features. Extensive experiments on ten real-world datasets, including eight small high-dimensional sets and two face-recognition collections, demonstrate that S^2FS consistently outperforms eight baselines in classification accuracy and clustering NMI, with visible interpretability of the selected features in facial images. The results underscore the value of integrating spatial directional information into separability measures, achieving robust performance with relatively few features and offering practical benefits for transparent, high-dimensional FDS applications. Overall, S^2FS provides a principled, efficient, and interpretable approach to feature selection in fuzzy decision settings, with strong implications for improved decision boundaries and real-world tasks such as face recognition.

Abstract

Feature selection is crucial for fuzzy decision systems (FDSs), as it identifies informative features and eliminates rule redundancy, thereby enhancing predictive performance and interpretability. Most existing methods either fail to directly align evaluation criteria with learning performance or rely solely on non-directional Euclidean distances to capture relationships among decision classes, which limits their ability to clarify decision boundaries. However, the spatial distribution of instances has a potential impact on the clarity of such boundaries. Motivated by this, we propose Spatially-aware Separability-driven Feature Selection (S$^2$FS), a novel framework for FDSs guided by a spatially-aware separability criterion. This criterion jointly considers within-class compactness and between-class separation by integrating scalar-distances with spatial directional information, providing a more comprehensive characterization of class structures. S$^2$FS employs a forward greedy strategy to iteratively select the most discriminative features. Extensive experiments on ten real-world datasets demonstrate that S$^2$FS consistently outperforms eight state-of-the-art feature selection algorithms in both classification accuracy and clustering performance, while feature visualizations further confirm the interpretability of the selected features.

Figures (18)

• Figure 1: Spatial distribution of instances from two decision classes. (a) Isotropic distribution. (b) Two-sided concentrated distribution.
• Figure 2: An illustration of constructing within-class compactness from direction perspective
• Figure 3: An illustration of conflicts among multiple decision classes in constructing within-class compactness
• Figure 4: An illustration of conflicts among multiple decision classes in constructing between-class separation
• Figure 5: Overall S$^2$FS framework and underlying processing