Sensitivity of low-rank matrix recovery

TL;DR

The paper investigates the first-order sensitivity of recovering a rank-$r$ matrix from linear measurements and of approximating a matrix by a rank-$r$ matrix, using a Riemannian geometric framework on the rank-$r$ manifold. It derives a closed-form bound for the low-rank approximation case, $\kappa_{\mathrm{approximation}}(A,Y)=\frac{1}{1-\sigma_{r+1}/\sigma_r}$, and provides a practical algorithm to compute the recovery condition number $\kappa_{\mathrm{recovery}}(A,Y)$ via the Riemannian Hessian and second fundamental form, with complexity $\mathcal{O}(\varphi s^3)$ for structured sensing where $s=(m+n-r)r$ and $\ell=\varphi s$. The approach yields explicit expressions for the second fundamental form and Weingarten map under affine sensing and demonstrates, through numerical experiments, that oversampling reduces conditioning ill-posedness and that the condition number grows as the singular-value gap $\sigma_r-\sigma_{r+1}$ shrinks. The results provide design guidance for sensing strategies in applications like collaborative filtering and image inpainting, highlighting the practical impact of curvature on stability in low-rank recovery and approximation.

Abstract

We characterize the first-order sensitivity of approximately recovering a low-rank matrix from linear measurements, a standard problem in compressed sensing. A special case covered by our analysis is approximating an incomplete matrix by a low-rank matrix. We give an algorithm for computing the associated condition number and demonstrate experimentally how the number of linear measurements affects it. In addition, we study the condition number of the rank-r matrix approximation problem. It measures in the Frobenius norm by how much an infinitesimal perturbation to an arbitrary input matrix is amplified in the movement of its best rank-r approximation. We give an explicit formula for the condition number, which shows that it does depend on the relative singular value gap between the rth and (r+1)th singular values of the input matrix.

Figures (3)

• Figure 1: The simplified geometry of this paper. On the right is the submanifold $\mathcal{R}_r\subset \mathcal{M}_r$ of identifiable rank-$r$ matrices, and on the left is the submanifold $\mathcal{S}_r\subset \mathcal{I}_r$ of sensed identifiable matrices. If the affine linear map $L$ is generic, then it restricts to a diffeomorphism $\mathcal{R}_r\to\mathcal{S}_r$. The low-rank matrix recovery problem consists of two steps: (i) projecting the data point $A\in\mathbb{R}^\ell$ to $X\in\mathcal{S}_r$ with $\Pi$, and (ii) finding $Y\in\mathcal{R}_r$ with $L(Y)=X$. Therefore, the sensitivity of the output $Y$ with respect to the input perturbation $A' - A$ depends on the combined impact of (i) the curvature of $\mathcal{S}_r$ which causes $X$ to move to $X' \in \mathcal{S}_r$ as $A$ moves to $A'$, and (ii) the sensitivity of inverting $L\mid_{\mathcal{R}_r}$ which forces $Y$ to move to $Y' \in \mathcal{R}_r$ as $X$ moves to $X'$.
• Figure 2: The picture shows how curvature affects the sensitivity of computing closest points on nonlinear objects. In this case, the curvature of the parabola amplifies the error $\Delta A$ in $A$. The amplification of errors is determined by the eigenvalues of the Riemannian Hessian $H_{A,X}$.
• Figure 3: The base-$10$ logarithm of the condition number $\kappa_\textrm{recovery}$ for various combinations of the oversampling factor $\phi$ and the distance signed $t$.

Theorems & Definitions (11)

• Proposition 1
• Remark 1
• Theorem 2
• Lemma 1
• proof : Proof of \ref{['main_thm1']}
• Remark 2
• Example 1
• Lemma 2
• Remark 3
• Proposition 3