Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unsupervised Feature Selection via Nonnegative Orthogonal Constrained Regularized Minimization

Yan Li, Defeng Sun, Liping Zhang

TL;DR

A novel unsupervised feature selection model based on regularized minimization with nonnegative orthogonal constraints is established, which has advantages of embedding feature selection into the nonnegative spectral clustering and preventing overfitting.

Abstract

Unsupervised feature selection has drawn wide attention in the era of big data since it is a primary technique for dimensionality reduction. However, many existing unsupervised feature selection models and solution methods were presented for the purpose of application, and lack of theoretical support, e.g., without convergence analysis. In this paper, we first establish a novel unsupervised feature selection model based on regularized minimization with nonnegative orthogonal constraints, which has advantages of embedding feature selection into the nonnegative spectral clustering and preventing overfitting. An effective inexact augmented Lagrangian multiplier method is proposed to solve our model, which adopts the proximal alternating minimization method to solve subproblem at each iteration. We show that the sequence generated by our method globally converges to a Karush-Kuhn-Tucker point of our model. Extensive numerical experiments on popular datasets demonstrate the stability and robustness of our method. Moreover, comparison results of algorithm performance show that our method outperforms some existing state-of-the-art methods.

Unsupervised Feature Selection via Nonnegative Orthogonal Constrained Regularized Minimization

TL;DR

A novel unsupervised feature selection model based on regularized minimization with nonnegative orthogonal constraints is established, which has advantages of embedding feature selection into the nonnegative spectral clustering and preventing overfitting.

Abstract

Unsupervised feature selection has drawn wide attention in the era of big data since it is a primary technique for dimensionality reduction. However, many existing unsupervised feature selection models and solution methods were presented for the purpose of application, and lack of theoretical support, e.g., without convergence analysis. In this paper, we first establish a novel unsupervised feature selection model based on regularized minimization with nonnegative orthogonal constraints, which has advantages of embedding feature selection into the nonnegative spectral clustering and preventing overfitting. An effective inexact augmented Lagrangian multiplier method is proposed to solve our model, which adopts the proximal alternating minimization method to solve subproblem at each iteration. We show that the sequence generated by our method globally converges to a Karush-Kuhn-Tucker point of our model. Extensive numerical experiments on popular datasets demonstrate the stability and robustness of our method. Moreover, comparison results of algorithm performance show that our method outperforms some existing state-of-the-art methods.
Paper Structure (13 sections, 5 theorems, 73 equations, 4 figures, 3 tables, 2 algorithms)

This paper contains 13 sections, 5 theorems, 73 equations, 4 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. A New Unsupervised Feature Selection Model
  4. Algorithm Description of Our Inexact ALM Method
  5. PAM for Augmented Lagrangian Subproblem
  6. Convergence Analysis for Algorithm \ref{['algo2']}
  7. Convergence Analysis of Our Inexact ALM Method
  8. Experiment Study
  9. Algorithmic Performance
  10. Stability Analysis
  11. Robustness Analysis
  12. Parameter Sensitivity Analysis
  13. conclusion

Key Result

Theorem 6

Set parameters $r> 1, \rho^1> 0$ in Algorithm algo1. For each $k\in\mathbb{N}$, we have the sequence $\{T^{k,j}\}_{j\in\mathbb{N}}$ produced by Algorithm algo2 converges and

Figures (4)

  • Figure 1: Stability curves over lung and Isolet.
  • Figure 2: Robustness comparison to data perturbation between our method and other iterative methods on lung.
  • Figure 3: Robustness comparison to data perturbation between our method and other iterative methods on Isolet.
  • Figure 4: Performance with different $\alpha$, $\beta$, $\gamma$ values on Isolet and lung with a grid search strategy.

Theorems & Definitions (10)

  • Definition 1
  • Definition 2
  • Remark 3: Closedness of $\partial f$
  • Remark 4
  • Remark 5
  • Theorem 6
  • Theorem 7
  • Lemma 8
  • Lemma 9
  • Theorem 10