Semi-supervised Symmetric Non-negative Matrix Factorization with Low-Rank Tensor Representation

TL;DR

A novel semi-supervised SNMF model is proposed by seeking low-rank representation for the tensor synthesized by the pairwise constraint matrix and a similarity matrix obtained by the product of the embedding matrix and its transpose, which could strengthen those two matrices simultaneously from a global perspective.

Abstract

Semi-supervised symmetric non-negative matrix factorization (SNMF) utilizes the available supervisory information (usually in the form of pairwise constraints) to improve the clustering ability of SNMF. The previous methods introduce the pairwise constraints from the local perspective, i.e., they either directly refine the similarity matrix element-wisely or restrain the distance of the decomposed vectors in pairs according to the pairwise constraints, which overlook the global perspective, i.e., in the ideal case, the pairwise constraint matrix and the ideal similarity matrix possess the same low-rank structure. To this end, we first propose a novel semi-supervised SNMF model by seeking low-rank representation for the tensor synthesized by the pairwise constraint matrix and a similarity matrix obtained by the product of the embedding matrix and its transpose, which could strengthen those two matrices simultaneously from a global perspective. We then propose an enhanced SNMF model, making the embedding matrix tailored to the above tensor low-rank representation. We finally refine the similarity matrix by the strengthened pairwise constraints. We repeat the above steps to continuously boost the similarity matrix and pairwise constraint matrix, leading to a high-quality embedding matrix. Extensive experiments substantiate the superiority of our method. The code is available at https://github.com/JinaLeejnl/TSNMF.

Figures (11)

• Figure 1: Workflow of TSNMF. Given the data matrix $X$, the initial similarity matrix $S$ is obtained, and the supervisory information is converted into the initial pairwise constraint matrix $Z$, where the yellow line represents must-link and the blue line represents cannot-link. The following processes is mainly divided into three steps: (a) Enhanced SNMF, (b) Tensor low-rank representation and (c) using $Z$ to adjust $S$, where $V^*$ represents the embedding matrix obtained from step (a), $A$ is obtained from the product of $V^*$ with its transpose, and $\bm{\mathcal{C}}$ represents the tensor synthesized by similarity matrix $S$ and pairwise constraint matrix $Z$.
• Figure 2: Comparisons of the accuracy of different methods on 6 datasets with different numbers of classes. The legends of the 6 subfigures are consistent and shown in \ref{['ACC_0.1_BinAlpha']}.
• Figure 3: Comparisons of the NMI of different methods on 6 datasets with different number of classes. The legends of the 6 subfigures are consistent and shown in \ref{['NMI_0.1_BinAlpha']}.
• Figure 4: Accuracy of SNMF-based semi-supervised methods on different amounts of supervisory information. The legends of the 6 subfigures are consistent and shown in \ref{['ACC_label_BinAlpha']}.
• Figure 5: NMI of SNMF-based semi-supervised methods on different amounts of supervisory information. The legends of the 6 subfigures are consistent and shown in \ref{['NMI_label_BinAlpha']}.