High Rank Matrix Completion via Grassmannian Proxy Fusion

TL;DR

High-rank matrix completion (HRMC) seeks to fill missing entries in data where columns inhabit a union of subspaces. The authors introduce GrassFusion, a Grassmannian-optimization method that assigns each incomplete vector to a proxy subspace and minimizes a joint objective combining a chordal-consistency term $d_c^2({\boldsymbol{x}}_i^\Omega, {\boldsymbol{U}}_i)$ and a geodesic-consensus term $d_g^2({\boldsymbol{U}}_i,{\boldsymbol{U}}_j)$ with penalty $\lambda$, enabling clustering without knowing the number of subspaces $K$. Proxies are optimized via Grassmannian gradient descent with Armijo steps, after which standard clustering (spectral or k-means) is applied to obtain subspace groups, followed by LRMC-based completion and subspace identification via SVD. Experiments on synthetic and real datasets show GrassFusion matches leading methods at high sampling rates and significantly outperforms them at low sampling, narrowing the gap toward the information-theoretic sampling limit for HRMC. The work provides a theoretically grounded, robust approach with local convergence guarantees and practical applicability to noisy, sparse HRMC problems.

Abstract

This paper approaches high-rank matrix completion (HRMC) by filling missing entries in a data matrix where columns lie near a union of subspaces, clustering these columns, and identifying the underlying subspaces. Current methods often lack theoretical support, produce uninterpretable results, and require more samples than theoretically necessary. We propose clustering incomplete vectors by grouping proxy subspaces and minimizing two criteria over the Grassmannian: (a) the chordal distance between each point and its corresponding subspace and (b) the geodesic distances between subspaces of all data points. Experiments on synthetic and real datasets demonstrate that our method performs comparably to leading methods in high sampling rates and significantly better in low sampling rates, thus narrowing the gap to the theoretical sampling limit of HRMC.

Figures (6)

• Figure 1: GrassFusion: Assigning each point with incomplete data to a unique subspace with complete data and then minimizing two criteria over the Grassmannian: (a) the chordal distance between each point and its corresponding subspace, ensuring the subspace can potentially complete the observed vector, and (b) the geodesic distances between subspaces of all data points, encouraging subspaces of similar points to merge, effectively representing the same space. Once this optimization is completed, we cluster the proxy subspaces using standard methods like spectral clustering.
• Figure 2: The semi-spheres represent the Grassmannian $\mathbb G({{{\rm m}}}, {{{\rm r}}})$, where each point ${{\mathbb{U}}}_{{{\rm i}}}$ denotes a subspace (in the case of $\mathbb G(3, 1)$, this is represented by the line extending from the origin to ${{\mathbb{U}}}_{{{\rm i}}}$). Left: The chordal distance ${{d}}_c({{\boldsymbol{{\rm x}}}}_{{{\rm i}}}^{{\Omega}}, {{\boldsymbol{{\rm U}}}}_{{{\rm i}}})$ serves as an informal measure of the distance between the subspace ${{\mathbb{U}}}_{{{\rm i}}}$ and the incomplete point ${{\boldsymbol{{\rm x}}}}_{{{\rm i}}}^{{\Omega}}$. This illustration is intended for intuitive purposes only, as ${{\mathbb{X}}}_{{{\rm i}}}^0$ may not reside on the same Grassmannian, and the chordal distance should not be equated with geodesic distance. Right: The geodesic distance ${{d}}_g({{\boldsymbol{{\rm U}}}}_{{{\rm i}}}, {{\boldsymbol{{\rm U}}}}_{{{\rm j}}})$ measures the distance over the Grassmannian between ${{\mathbb{U}}}_{{{\rm i}}}$ and ${{\mathbb{U}}}_{{{\rm j}}}$. Bottom: The Euclidean gradient vector ${{\boldsymbol{\nabla}}}_{{{\rm i}}}$ deviates from the Grassmann manifold; thus, each geodesic step must be adjusted to account for the curvature of the Grassmannian, as specified by \ref{['stepEq']}.
• Figure 3: ${{\lambda}} \geq 0$ in \ref{['stiefelEq']} regulates how clusters fuse together. If ${{\lambda}}=0$, each point is assigned to a subspace that exactly contains it (overfitting). The larger ${{\lambda}}$, the more we penalize subspaces being apart, which results in subspaces getting closer to form fewer clusters. The extreme case ${{\lambda}}=\infty$ is the special case of PCA and LRMC, where only one subspace is allowed to explain all data.
• Figure 4: Clustering accuracy, sum of chordal distance, and sum of geodesic distance across training iterations. Five key timestamps (marked by red dots at iterations 0, 600, 4800, 6900, and 9900) were selected and visualized using GrassCaré li2024grasscare, a tool for visualizing subspaces on the Grassmannian. The visualization shows how the subspace proxies (yellow and blue dots) interact with the objective function. The red star indicates the ground-truth subspace used in the initialization.
• Figure 5: Clustering error (average over 10 trials) as a function of sampling rate for different synthetic settings. The left-most vertical line at ${{{\rm p}}}^\star=({{{\rm r}}}+1)/\min({{{\rm m}}},{{{\rm n}}})$ represents the information-theoretic sampling limit pimentel2016information. That is, HRMC is impossible for any ${{{\rm p}}}<{{{\rm p}}}^\star$, and is theoretically possible for any ${{{\rm p}}} \geq {{{\rm p}}}^\star$ (for example, with a brute-force combinatorial algorithm). The right-most vertical line indicates the limit of the state-of-the-art. The center vertical line indicates the sampling limit of our approach, which shortens the gap towards the theoretical limit.

Theorems & Definitions (2)

• Theorem 3.1
• proof