Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

i-IF-Learn: Iterative Feature Selection and Unsupervised Learning for High-Dimensional Complex Data

Chen Ma, Wanjie Wang, Shuhao Fan

Abstract

Unsupervised learning of high-dimensional data is challenging due to irrelevant or noisy features obscuring underlying structures. It's common that only a few features, called the influential features, meaningfully define the clusters. Recovering these influential features is helpful in data interpretation and clustering. We propose i-IF-Learn, an iterative unsupervised framework that jointly performs feature selection and clustering. Our core innovation is an adaptive feature selection statistic that effectively combines pseudo-label supervision with unsupervised signals, dynamically adjusting based on intermediate label reliability to mitigate error propagation common in iterative frameworks. Leveraging low-dimensional embeddings (PCA or Laplacian eigenmaps) followed by $k$-means, i-IF-Learn simultaneously outputs influential feature subset and clustering labels. Numerical experiments on gene microarray and single-cell RNA-seq datasets show that i-IF-Learn significantly surpasses classical and deep clustering baselines. Furthermore, using our selected influential features as preprocessing substantially enhances downstream deep models such as DeepCluster, UMAP, and VAE, highlighting the importance and effectiveness of targeted feature selection.

i-IF-Learn: Iterative Feature Selection and Unsupervised Learning for High-Dimensional Complex Data

Abstract

Unsupervised learning of high-dimensional data is challenging due to irrelevant or noisy features obscuring underlying structures. It's common that only a few features, called the influential features, meaningfully define the clusters. Recovering these influential features is helpful in data interpretation and clustering. We propose i-IF-Learn, an iterative unsupervised framework that jointly performs feature selection and clustering. Our core innovation is an adaptive feature selection statistic that effectively combines pseudo-label supervision with unsupervised signals, dynamically adjusting based on intermediate label reliability to mitigate error propagation common in iterative frameworks. Leveraging low-dimensional embeddings (PCA or Laplacian eigenmaps) followed by -means, i-IF-Learn simultaneously outputs influential feature subset and clustering labels. Numerical experiments on gene microarray and single-cell RNA-seq datasets show that i-IF-Learn significantly surpasses classical and deep clustering baselines. Furthermore, using our selected influential features as preprocessing substantially enhances downstream deep models such as DeepCluster, UMAP, and VAE, highlighting the importance and effectiveness of targeted feature selection.
Paper Structure (37 sections, 3 theorems, 53 equations, 5 figures, 12 tables, 8 algorithms)

This paper contains 37 sections, 3 theorems, 53 equations, 5 figures, 12 tables, 8 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Our Contribution
  4. Organization
  5. Algorithm: Iterative High-Dimensional Clustering
  6. Initialization
  7. Novel Iterative Screening Statistic
  8. Feature Selection by HCT
  9. Embedding and Clustering
  10. Stopping Criteria
  11. Model Assumptions and Theoretical Guarantee
  12. Real Datasets
  13. Gene Microarray Datasets
  14. Single-cell RNA Sequencing Datasets
  15. Statistical Significance
  16. ...and 22 more sections

Key Result

Theorem 3.1

Consider the estimated label $\hat{\ell}$, which can be the initial label $\hat{\ell}^{(0)}$ or the estimated label $\ell^{(t-1)}$ from last round. Denote $w_{ij} = E[\Sigma^{-1/2}X_{ij}]$ as the expectation for data point $i$ on feature $j$, and the overall mean $\bar{w}_j=\frac{1}{n}\sum_{i=1}^n w Then our weight selection satisfies $P(w^{(t)} \geq 1-p^{-2})\geq 1-p^{-2}$. Furthermore, with prob

Figures (5)

  • Figure 1: Eigenvector projection of data points in 3D space using three clustering pipelines.
  • Figure 2: Overview of the i-IF-Learn framework.
  • Figure 3: Left: FDR of feature selection step versus signal strength for signals in $I_w$. Right: the clustering accuracy versus signal strength for signals in $I_w$.
  • Figure 4: Left: clustering accuracy versus dimension $p$ in the $p$-Sweep experiment. Right: clustering accuracy versus the power $a$ in the $\mu$-Sweep experiment.
  • Figure 5: Comparison of feature selection performance under increasing weak signal strength ($\tau_w$). Each subplot reports a different metric: (a) False Positive Rate (FPR), (b) True Positive Rate (TPR), (c) False Discovery Rate (FDR), and (d) True Discovery Rate (TDR).

Theorems & Definitions (3)

  • Theorem 3.1
  • Theorem 3.2
  • Lemma A.1