Tuning-Free Structured Sparse PCA via Deep Unfolding Networks
Long Chen, Xianchao Xiu
TL;DR
The paper tackles tuning-heavy sparse PCA for unsupervised feature selection by introducing a structured sparsity formulation combining $\ell_1$ and $\ell_{2,1}$ norms. It then develops SPCA-Net, a deep unfolding network that converts ADMM iterations into trainable layers, learning regularization and penalty parameters to eliminate manual tuning. The method achieves superior performance on multiple benchmarks (ACC and NMI) compared with state-of-the-art approaches, while requiring fewer iterations (3–5 stages) than conventional optimization. This tuning-free approach improves interpretability and computational efficiency, with solid potential for broader applicability in UFS tasks.
Abstract
Sparse principal component analysis (PCA) is a well-established dimensionality reduction technique that is often used for unsupervised feature selection (UFS). However, determining the regularization parameters is rather challenging, and conventional approaches, including grid search and Bayesian optimization, not only bring great computational costs but also exhibit high sensitivity. To address these limitations, we first establish a structured sparse PCA formulation by integrating $\ell_1$-norm and $\ell_{2,1}$-norm to capture the local and global structures, respectively. Building upon the off-the-shelf alternating direction method of multipliers (ADMM) optimization framework, we then design an interpretable deep unfolding network that translates iterative optimization steps into trainable neural architectures. This innovation enables automatic learning of the regularization parameters, effectively bypassing the empirical tuning requirements of conventional methods. Numerical experiments on benchmark datasets validate the advantages of our proposed method over the existing state-of-the-art methods. Our code will be accessible at https://github.com/xianchaoxiu/SPCA-Net.