Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cascaded two-stage feature clustering and selection via separability and consistency in fuzzy decision systems

Yuepeng Chen, Weiping Ding, Hengrong Ju, Jiashuang Huang, Tao Yin

TL;DR

The paper introduces FCSSC, a cascaded two-stage feature clustering and selection framework for fuzzy decision systems that integrates a clustering-based forward search with a novel fusion metric combining global separability and local consistency. By first clustering features with fuzzy C-Means and then selecting features via SIG scores within clusters, FCSSC achieves superior classification accuracy while using fewer features across 18 public datasets and a schizophrenia rs-fMRI dataset. Key ideas include defining GS_B(D) through intra-class cohesion and inter-class separation and LC_B(D) via adaptive fuzzy neighborhood relations, balanced by a parameter β, and selecting features to maximize the discriminative gamma score. The work demonstrates significant performance gains over six benchmarks and highlights potential for biomedical applications, with future directions including kernel-based expansions and multi-view data handling.

Abstract

Feature selection is a vital technique in machine learning, as it can reduce computational complexity, improve model performance, and mitigate the risk of overfitting. However, the increasing complexity and dimensionality of datasets pose significant challenges in the selection of features. Focusing on these challenges, this paper proposes a cascaded two-stage feature clustering and selection algorithm for fuzzy decision systems. In the first stage, we reduce the search space by clustering relevant features and addressing inter-feature redundancy. In the second stage, a clustering-based sequentially forward selection method that explores the global and local structure of data is presented. We propose a novel metric for assessing the significance of features, which considers both global separability and local consistency. Global separability measures the degree of intra-class cohesion and inter-class separation based on fuzzy membership, providing a comprehensive understanding of data separability. Meanwhile, local consistency leverages the fuzzy neighborhood rough set model to capture uncertainty and fuzziness in the data. The effectiveness of our proposed algorithm is evaluated through experiments conducted on 18 public datasets and a real-world schizophrenia dataset. The experiment results demonstrate our algorithm's superiority over benchmarking algorithms in both classification accuracy and the number of selected features.

Cascaded two-stage feature clustering and selection via separability and consistency in fuzzy decision systems

TL;DR

The paper introduces FCSSC, a cascaded two-stage feature clustering and selection framework for fuzzy decision systems that integrates a clustering-based forward search with a novel fusion metric combining global separability and local consistency. By first clustering features with fuzzy C-Means and then selecting features via SIG scores within clusters, FCSSC achieves superior classification accuracy while using fewer features across 18 public datasets and a schizophrenia rs-fMRI dataset. Key ideas include defining GS_B(D) through intra-class cohesion and inter-class separation and LC_B(D) via adaptive fuzzy neighborhood relations, balanced by a parameter β, and selecting features to maximize the discriminative gamma score. The work demonstrates significant performance gains over six benchmarks and highlights potential for biomedical applications, with future directions including kernel-based expansions and multi-view data handling.

Abstract

Feature selection is a vital technique in machine learning, as it can reduce computational complexity, improve model performance, and mitigate the risk of overfitting. However, the increasing complexity and dimensionality of datasets pose significant challenges in the selection of features. Focusing on these challenges, this paper proposes a cascaded two-stage feature clustering and selection algorithm for fuzzy decision systems. In the first stage, we reduce the search space by clustering relevant features and addressing inter-feature redundancy. In the second stage, a clustering-based sequentially forward selection method that explores the global and local structure of data is presented. We propose a novel metric for assessing the significance of features, which considers both global separability and local consistency. Global separability measures the degree of intra-class cohesion and inter-class separation based on fuzzy membership, providing a comprehensive understanding of data separability. Meanwhile, local consistency leverages the fuzzy neighborhood rough set model to capture uncertainty and fuzziness in the data. The effectiveness of our proposed algorithm is evaluated through experiments conducted on 18 public datasets and a real-world schizophrenia dataset. The experiment results demonstrate our algorithm's superiority over benchmarking algorithms in both classification accuracy and the number of selected features.
Paper Structure (14 sections, 4 theorems, 25 equations, 7 figures, 5 tables, 1 algorithm)

This paper contains 14 sections, 4 theorems, 25 equations, 7 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. PRELIMINARIES
  3. Fuzzy C-Means algorithm
  4. Fuzzy neighborhood rough set
  5. The proposed method
  6. Feature clustering based on FCM
  7. Feature selection based on global separability and local consistency
  8. Global separability based on fuzzy membership
  9. Local consistency based on fuzzy similarity relation
  10. Feature selection via global separability and local consistency
  11. Experimental Analysis
  12. Experiment on the UCI and the Kent Ridge Biomedical Datasets
  13. Feature selection on the schizophrenia dataset
  14. Conclusion

Key Result

Proposition 1

In a fuzzy decision system $S$, $D_i \subseteq U/D$, $B \subseteq A$, then $0 \le \mathrm{DIC}_B(D) \le 1$.

Figures (7)

  • Figure 1: The framework of feature clustering and selection
  • Figure 2: The framework for measuring the significance of features
  • Figure 3: The average classification accuracy variations with parameter $\beta$ and the number of features.
  • Figure 4: The comparison of classification accuracy under three classifiers.
  • Figure 5: Classification performance with different features under the KNN classifier in three metrics.
  • ...and 2 more figures

Theorems & Definitions (5)

  • Proposition 1
  • Proposition 2
  • Proposition 3
  • Example 1
  • Proposition 4