Randomized coupled decompositions

TL;DR

The paper addresses the computational challenge of coupled decompositions by showing that CMF and CMTF problems can be solved via direct SVD-based rank-$k$ approximations, recast as the SVD of a single matrix $XY$ or $X_{(1)}Y$. It introduces randomized projection strategies to efficiently handle large-scale data, including a joint subspace projection that equilibrates contributions from all participating matrices, and enhancements using RSI and RBKI. The authors provide extensive numerical experiments on synthetic CMF/CMTF problems and demonstrate substantial speedups with comparable or improved accuracy, including novel applications to face recognition. The work also analyzes tensor formats (Tucker and CP) within CMTF, offering a provably minimal objective in the Tucker-based approach and practical ALS-based comparisons. Overall, the methods enable scalable, reliable coupled decompositions with real-world impact in high-dimensional data fusion tasks.

Abstract

Coupled decompositions are a widely used tool for data fusion. As the volume of data increases, so does the dimensionality of matrices and tensors, highlighting the need for more efficient coupled decomposition algorithms. This paper studies the problem of coupled matrix factorization (CMF), where two matrices represented in low-rank form share a common factor. Additionally, it explores coupled matrix and tensor factorization (CMTF), where a matrix and a tensor are represented in low-rank form, also sharing a common factor matrix. We show that these problems can be solved using a direct approach with singular value decomposition (SVD), rather than relying on an iterative method. Knowing that matrices coming from real-world applications are often very large, the computational cost can be substantial. To address this issue and improve the efficiency, we propose new techniques for randomizing these algorithms. This includes a novel strategy for selecting a projection subspace that takes into account the contribution from both matrices involved in the decomposition equally. We present extensive results of numerical tests that confirm the efficiency of our algorithms. Furthermore, as a novel approach and with a high success rate, we apply our randomized algorithms to the face recognition problem.

Figures (11)

• Figure 1: Graphical description of CMF.
• Figure 2: Comparison of two different approaches for computing the projection matrix $Q$ on $100$ randomly generated examples.
• Figure 3: Relative errors with the respect to the dimension of the projection subspace in the Algorithm \ref{['agm:CMFRBKI']} for SyntheticTest1.
• Figure 4: Relative errors with the respect to the dimension of the projection subspace in the Algorithm \ref{['agm:CMFRBKI']} for SyntheticTest2.
• Figure 5: RBKI with $\ell=1$ (on the left) and $\ell=2$ (on the right). Rank-revealing QR leads to a smaller oversampling parameter.

Theorems & Definitions (4)

• proof
• proof
• proof
• proof