A novel feature selection framework for incomplete data

TL;DR

This work tackles feature selection on incomplete data by introducing an alternating two-stage framework that integrates feature importance into the imputation process. The M-stage performs importance-weighted matrix completion using an ensemble of imputations, while the W-stage learns feature importance via a modified $bc$-reliefA algorithm from the imputed data, with iterative feedback between stages. Empirical results on artificial and real missing-value datasets show that EWMC+$bc$-reliefA consistently outperforms baselines, with significant improvements confirmed by Wilcoxon tests and robust performance across missing mechanisms and rates. The approach enhances both imputation quality and feature subset relevance, offering a practical pathway for reliable feature selection under incomplete data conditions.

Abstract

Feature selection on incomplete datasets is an exceptionally challenging task. Existing methods address this challenge by first employing imputation methods to complete the incomplete data and then conducting feature selection based on the imputed data. Since imputation and feature selection are entirely independent steps, the importance of features cannot be considered during imputation. However, in real-world scenarios or datasets, different features have varying degrees of importance. To address this, we propose a novel incomplete data feature selection framework that considers feature importance. The framework mainly consists of two alternating iterative stages: the M-stage and the W-stage. In the M-stage, missing values are imputed based on a given feature importance vector and multiple initial imputation results. In the W-stage, an improved reliefF algorithm is employed to learn the feature importance vector based on the imputed data. Specifically, the feature importance vector obtained in the current iteration of the W-stage serves as input for the next iteration of the M-stage. Experimental results on both artificially generated and real incomplete datasets demonstrate that the proposed method outperforms other approaches significantly.

Figures (6)

• Figure 1: Comparison results of eight methods on four datasets under MCAR.
• Figure 2: Comparison results of eight methods on four datasets under MNAR.
• Figure 3: Comparison results of eight methods on six datasets with real missing values.
• Figure 4: Accuracy of the proposed method on the Wine and Heart datasets under MCAR for different parameter settings.
• Figure 5: Accuracy of the proposed method on the Wine and Heart datasets under MNAR for different parameter settings.