Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Matrix Factorization with Adaptive Weights for Multi-View Clustering

Yasser Khalafaoui, Basarab Matei, Martino Lovisetto, Nistor Grozavu

TL;DR

This work addresses multi-view clustering with deep matrix factorization by introducing DMFAW, a weighted deep semi-NMF framework that performs simultaneous feature selection and local partition learning. A dynamic, PI-inspired update of the feature-selection parameter $p$ enhances stability and accelerates convergence, while a late-fusion scheme computes a robust consensus partition $G^*$ from per-view local partitions. The method is shown to achieve state-of-the-art clustering performance across six benchmark datasets, with notable gains on BBCSport and BBC, and to converge efficiently within few iterations. Overall, DMFAW offers a principled, performance-driven approach to feature relevance in deep MVC, with practical benefits for scalability and robustness in diverse data settings.

Abstract

Recently, deep matrix factorization has been established as a powerful model for unsupervised tasks, achieving promising results, especially for multi-view clustering. However, existing methods often lack effective feature selection mechanisms and rely on empirical hyperparameter selection. To address these issues, we introduce a novel Deep Matrix Factorization with Adaptive Weights for Multi-View Clustering (DMFAW). Our method simultaneously incorporates feature selection and generates local partitions, enhancing clustering results. Notably, the features weights are controlled and adjusted by a parameter that is dynamically updated using Control Theory inspired mechanism, which not only improves the model's stability and adaptability to diverse datasets but also accelerates convergence. A late fusion approach is then proposed to align the weighted local partitions with the consensus partition. Finally, the optimization problem is solved via an alternating optimization algorithm with theoretically guaranteed convergence. Extensive experiments on benchmark datasets highlight that DMFAW outperforms state-of-the-art methods in terms of clustering performance.

Deep Matrix Factorization with Adaptive Weights for Multi-View Clustering

TL;DR

This work addresses multi-view clustering with deep matrix factorization by introducing DMFAW, a weighted deep semi-NMF framework that performs simultaneous feature selection and local partition learning. A dynamic, PI-inspired update of the feature-selection parameter enhances stability and accelerates convergence, while a late-fusion scheme computes a robust consensus partition from per-view local partitions. The method is shown to achieve state-of-the-art clustering performance across six benchmark datasets, with notable gains on BBCSport and BBC, and to converge efficiently within few iterations. Overall, DMFAW offers a principled, performance-driven approach to feature relevance in deep MVC, with practical benefits for scalability and robustness in diverse data settings.

Abstract

Recently, deep matrix factorization has been established as a powerful model for unsupervised tasks, achieving promising results, especially for multi-view clustering. However, existing methods often lack effective feature selection mechanisms and rely on empirical hyperparameter selection. To address these issues, we introduce a novel Deep Matrix Factorization with Adaptive Weights for Multi-View Clustering (DMFAW). Our method simultaneously incorporates feature selection and generates local partitions, enhancing clustering results. Notably, the features weights are controlled and adjusted by a parameter that is dynamically updated using Control Theory inspired mechanism, which not only improves the model's stability and adaptability to diverse datasets but also accelerates convergence. A late fusion approach is then proposed to align the weighted local partitions with the consensus partition. Finally, the optimization problem is solved via an alternating optimization algorithm with theoretically guaranteed convergence. Extensive experiments on benchmark datasets highlight that DMFAW outperforms state-of-the-art methods in terms of clustering performance.

Paper Structure

This paper contains 33 sections, 2 theorems, 24 equations, 4 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Multi-view Clustering
  4. Deep Matrix Factorization
  5. Methodology
  6. Weighted Deep Matrix Factorization
  7. Adaptive Feature Selection
  8. Learning the Consensus Partition
  9. Optimization
  10. Subproblem of updating $G^*$
  11. Subproblem of updating $F_i^{(v)}$
  12. Subproblem of updating $G_i^{(v)} (i<m)$
  13. Subproblem of updating $G_m^{(v)}$
  14. Subproblem of updating $W^{(v)}$
  15. Subproblem of updating $M^{(v)}$
  16. ...and 18 more sections

Key Result

Theorem 1

If the matrix $U$, defined previously, has an economic rank-$k$ singular value decomposition form, then the optimization problem in Eq. eq:opt_cons has a closed-form solution defined as, where $V \in \mathbb{R}^{k\times k}$ and $S \in \mathbb{R}^{n\times k}$ are the right and left singular vectors respectively.

Figures (4)

  • Figure 1: A graphical representation of our proposed solution, DMFAW. The model integrates a weight matrix, denoted as $W^{(v)}$, for feature selection in each view's local clustering phase for each input matrix $X^{(v)}$. Then, a consensus partition matrix is generated based on local partitions $\{G_m^{(v)}\}_{v=1}^V$, permutation matrices $\{M^{(v)}\}_{v=1}^V$ and the average partition region $A$. Here, $F_i^{(v)}$ and $G_i^{(v)}$ represent the mapping and partition matrices of the $i$-th layer respectively, while $G^*$ represents the consensus partition matrix.
  • Figure 2: (a) Evolution of the objective value across iterations for Caltech101-7 Dataset. (b) Runtime in seconds, comparing our method to other baseline methods.
  • Figure 3: Sensitivity and clustering performance with different parameter settings on four datasets.
  • Figure 4: Visualization of pairwise similarity on BBCSport Dataset, using our method (a) and MVC-DMF-PA (b).

Theorems & Definitions (5)

  • Definition 1
  • Theorem 1
  • proof
  • Theorem 2
  • proof