Subspace-Constrained Quadratic Matrix Factorization: Algorithm and Applications

TL;DR

A subspace-constrained quadratic matrix factorization model designed to jointly learn key low-dimensional structures, including the tangent space, the normal subspace, and the quadratic form that links the tangent space to a low-dimensional representation is presented.

Abstract

Matrix Factorization has emerged as a widely adopted framework for modeling data exhibiting low-rank structures. To address challenges in manifold learning, this paper presents a subspace-constrained quadratic matrix factorization model. The model is designed to jointly learn key low-dimensional structures, including the tangent space, the normal subspace, and the quadratic form that links the tangent space to a low-dimensional representation. We solve the proposed factorization model using an alternating minimization method, involving an in-depth investigation of nonlinear regression and projection subproblems. Theoretical properties of the quadratic projection problem and convergence characteristics of the alternating strategy are also investigated. To validate our approach, we conduct numerical experiments on synthetic and real-world datasets. Results demonstrate that our model outperforms existing methods, highlighting its robustness and efficacy in capturing core low-dimensional structures.

Figures (9)

• Figure 1: Illustration of the matrix factorization landscape and the relationship between SQMF and prior works.
• Figure 2: The illustration depicts a fitted quadratic curve and the process of projecting noisy data onto the curve while minimizing the $\ell_2$ distance across various noise levels represented by $\sigma$. We demonstrate the performance of the Newton's method in the first row, the gradient algorithm in the second row, and the surrogate method in the third row.
• Figure 3: The comparison of the variation of the function value $\|X - Q^k M(\Theta^k) - c^k {\bf 1}^T \|_{\rm F}^2$ changes with the iteration steps $k$ for three different approaches: Gradient, Newton and Surrogate method.
• Figure 4: Visualization of the function $f(\tau)$ and its associated tangent space via fitting the samples in the vicinity of a two-dimensional sphere.. Left: Synthetic noisy data approximately distributed on a 2-dimensional sphere. Middle: The illustration of the estimated $f(\tau)$ by fitting the data in each local region. Right: Visualization of the tangent space.
• Figure 5: The illustration of the image's variation with the change of the latent coordinate for RSQMF $f(\tau)$ on the Frey Face and Mnist dataset, where the coordinates of $\tau$ represent the location of the corresponding image.

Theorems & Definitions (8)

• Proposition 1
• Proof
• Lemma 1
• Theorem 1
• Proposition 2
• Corollary 1
• Theorem 2
• Theorem 3