Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Causally-Aware Unsupervised Feature Selection Learning

Zongxin Shen, Yanyong Huang, Dongjie Wang, Minbo Ma, Fengmao Lv, Tianrui Li

TL;DR

A novel unsupervised feature selection method, called Causally-Aware UnSupErvised Feature Selection learning (CAUSE-FS), which introduces a novel causal regularizer that reweights samples to balance the confounding distribution of each treatment feature, thus achieving causal feature selection.

Abstract

Unsupervised feature selection (UFS) has recently gained attention for its effectiveness in processing unlabeled high-dimensional data. However, existing methods overlook the intrinsic causal mechanisms within the data, resulting in the selection of irrelevant features and poor interpretability. Additionally, previous graph-based methods fail to account for the differing impacts of non-causal and causal features in constructing the similarity graph, which leads to false links in the generated graph. To address these issues, a novel UFS method, called Causally-Aware UnSupErvised Feature Selection learning (CAUSE-FS), is proposed. CAUSE-FS introduces a novel causal regularizer that reweights samples to balance the confounding distribution of each treatment feature. This regularizer is subsequently integrated into a generalized unsupervised spectral regression model to mitigate spurious associations between features and clustering labels, thus achieving causal feature selection. Furthermore, CAUSE-FS employs causality-guided hierarchical clustering to partition features with varying causal contributions into multiple granularities. By integrating similarity graphs learned adaptively at different granularities, CAUSE-FS increases the importance of causal features when constructing the fused similarity graph to capture the reliable local structure of data. Extensive experimental results demonstrate the superiority of CAUSE-FS over state-of-the-art methods, with its interpretability further validated through feature visualization.

Causally-Aware Unsupervised Feature Selection Learning

TL;DR

A novel unsupervised feature selection method, called Causally-Aware UnSupErvised Feature Selection learning (CAUSE-FS), which introduces a novel causal regularizer that reweights samples to balance the confounding distribution of each treatment feature, thus achieving causal feature selection.

Abstract

Unsupervised feature selection (UFS) has recently gained attention for its effectiveness in processing unlabeled high-dimensional data. However, existing methods overlook the intrinsic causal mechanisms within the data, resulting in the selection of irrelevant features and poor interpretability. Additionally, previous graph-based methods fail to account for the differing impacts of non-causal and causal features in constructing the similarity graph, which leads to false links in the generated graph. To address these issues, a novel UFS method, called Causally-Aware UnSupErvised Feature Selection learning (CAUSE-FS), is proposed. CAUSE-FS introduces a novel causal regularizer that reweights samples to balance the confounding distribution of each treatment feature. This regularizer is subsequently integrated into a generalized unsupervised spectral regression model to mitigate spurious associations between features and clustering labels, thus achieving causal feature selection. Furthermore, CAUSE-FS employs causality-guided hierarchical clustering to partition features with varying causal contributions into multiple granularities. By integrating similarity graphs learned adaptively at different granularities, CAUSE-FS increases the importance of causal features when constructing the fused similarity graph to capture the reliable local structure of data. Extensive experimental results demonstrate the superiority of CAUSE-FS over state-of-the-art methods, with its interpretability further validated through feature visualization.

Paper Structure

This paper contains 27 sections, 2 theorems, 29 equations, 7 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Causally-Aware Unsupervised Feature Selection Learning
  4. Notations
  5. Generalized Causal Regression Model for UFS
  6. Multi-Granular Adaptive Graph Learning
  7. Optimization
  8. Updating $\bm{W}$ by Fixing Others
  9. Updating $\bm{S}^{(m)}$ by Fixing Others
  10. Updating $\bm{G}$ by Fixing Others
  11. Updating $\bm{F}$ by Fixing Others
  12. Updating $\bm{\mu}$ by Fixing Others
  13. Convergence and Complexity Analysis
  14. Convergence Analysis
  15. Time Complexity Analysis
  16. ...and 12 more sections

Key Result

Lemma 1

For any two nonzero vectors $\bm{a}$ and $\bm{b}$, the following inequality is satisfied:

Figures (7)

  • Figure 1: The examples of spurious correlation between features and clustering labels caused by confounders (left), and the unreliable links in the similarity graph result from ignoring the importance of different features (right).
  • Figure 2: The framework of the proposed CAUFS-FS.
  • Figure 3: ACC of different methods across various numbers of selected features on six datasets.
  • Figure 4: NMI of different methods across various numbers of selected features on six datasets.
  • Figure 5: Visualization of features selected by the runner-up method (the first row) and CAUSE-FS (the second row) on COIL20, Jaffe, and USPS datasets.
  • ...and 2 more figures

Theorems & Definitions (3)

  • Lemma 1
  • Theorem 1
  • proof