Stable Rank and Intrinsic Dimension of Real and Complex Matrices
Ilse C. F. Ipsen, Arvind K. Saibaba
TL;DR
This work broadens the toolkit for quantifying matrix information beyond classical rank by examining stable rank $\operatorname{sr}(A)$ and intrinsic dimension $\operatorname{intdim}(A)$, and then unifies these notions through the $p$-stable rank $\operatorname{sr}_p$ built from Schatten norms. The authors demonstrate that $\operatorname{sr}$ and $\operatorname{intdim}$ behave differently under basic operations (deletions, sums, products, and nonsingular changes), showing that many rank inequalities fail for $\operatorname{sr}$ while $\operatorname{intdim}$ satisfies a limited subset. They extend the framework to $p$-stable ranks, proving sum and product inequalities and establishing conditioning properties, with special emphasis on PSD and Hermitian PSD cases where bounds are sharper. The results yield robust, continuous measures that are applicable to randomized algorithms, covariance estimation, deep networks, and collaborative filtering, providing deeper insight into the information content and sensitivity of matrices in these domains.
Abstract
The notion of `stable rank' of a matrix is central to the analysis of randomized matrix algorithms, covariance estimation, deep neural networks, and recommender systems. We compare the properties of the stable rank and intrinsic dimension of real and complex matrices to those of the classical rank. Basic proofs and examples illustrate that the stable rank does not satisfy any of the fundamental rank properties, while the intrinsic dimension satisfies a few. In particular, the stable rank and intrinsic dimension of a submatrix can exceed those of the original matrix; adding a Hermitian positive semi-definite matrix can lower the intrinsic dimension of the sum; and multiplication by a nonsingular matrix can drastically change the stable rank and the intrinsic dimension. We generalize the concept of stable rank to the p-stable in any Schatten p-norm, thereby unifying the concepts of stable rank and intrinsic dimension: The stable rank is the 2-stable rank, while the intrinsic dimension is the 1-stable rank of a Hermitian positive semi-definite matrix. We derive sum and product inequalities for the pth root of the p-stable rank, and show that it is well-conditioned in the norm-wise absolute sense. The conditioning improves if the matrix and the perturbation are Hermitian positive semi-definite.