Input Guided Multiple Deconstruction Single Reconstruction neural network models for Matrix Factorization

TL;DR

The paper addresses high-dimensional data by learning non-negative, low-rank embeddings via two deep neural architectures, IG-MDSR-NMF and IG-MDSR-RNMF, that enforce a true NMF-like factorization or a relaxed RNMF variant within a deconstruction–single reconstruction framework guided by the input. The authors demonstrate that this input-guided, multi-layer decomposition yields a unique factor pair $(\textbf{B},\textbf{W})$ with $\hat{\textbf{X}} = \textbf{BW}$ while preserving local data structure, as measured by trustworthiness, and improving downstream classification and clustering across five diverse datasets. They provide comprehensive comparisons against nine state-of-the-art dimension reduction methods, showing superior or competitive performance in both embedding quality and downstream tasks, and they present convergence analyses confirming reliable optimization. The work advances scalable, non-negative dimension reduction by integrating principles of NMF with deep learning, offering robust latent representations applicable to various data regimes and practical analytics workflows.

Abstract

Referring back to the original text in the course of hierarchical learning is a common human trait that ensures the right direction of learning. The models developed based on the concept of Non-negative Matrix Factorization (NMF), in this paper are inspired by this idea. They aim to deal with high-dimensional data by discovering its low rank approximation by determining a unique pair of factor matrices. The model, named Input Guided Multiple Deconstruction Single Reconstruction neural network for Non-negative Matrix Factorization (IG-MDSR-NMF), ensures the non-negativity constraints of both factors. Whereas Input Guided Multiple Deconstruction Single Reconstruction neural network for Relaxed Non-negative Matrix Factorization (IG-MDSR-RNMF) introduces a novel idea of factorization with only the basis matrix adhering to the non-negativity criteria. This relaxed version helps the model to learn more enriched low dimensional embedding of the original data matrix. The competency of preserving the local structure of data in its low rank embedding produced by both the models has been appropriately verified. The superiority of low dimensional embedding over that of the original data justifying the need for dimension reduction has been established. The primacy of both the models has also been validated by comparing their performances separately with that of nine other established dimension reduction algorithms on five popular datasets. Moreover, computational complexity of the models and convergence analysis have also been presented testifying to the supremacy of the models.

Figures (25)

• Figure 1: The architecture of IG-MDSR-NMF
• Figure 2: Trustworthiness scores of ten dimension reduction techniques including IG-MDSR-NMF.
• Figure 3: Trustworthiness scores of ten dimension reduction techniques including IG-MDSR-RNMF.
• Figure 4: Performance scores of the classification algorithms on the dimensionally reduced dataset instances of the GLRC dataset by IG-MDSR-NMF and nine other dimension reduction techniques along with the original data.
• Figure 5: Performance scores of the classification algorithms on the dimensionally reduced dataset instances of the ONP dataset by IG-MDSR-NMF and nine other dimension reduction techniques along with the original data.