Riemannian optimization using three different metrics for Hermitian PSD fixed-rank constraints: an extended version

TL;DR

The paper analyzes three complementary approaches to optimize a real-valued function under Hermitian PSD fixed-rank constraints: the Burer–Monteiro factorization, Riemannian optimization on the embedded manifold, and Riemannian optimization on a quotient manifold. It shows that the Burer–Monteiro CG method corresponds to RCG on the quotient $(\mathbb{C}^{n\times p}_*/\mathcal{O}_p, g^1)$, while the embedded-geometry CG aligns with RCG on the quotient with metric $g^3$, enabling a unified quotient-based framework. A detailed Rayleigh-quotient analysis compares the condition numbers of the Riemannian Hessian across the three metrics $g^1,g^2,g^3$, revealing that the Bures–Wasserstein metric $g^1$ can exhibit unbounded conditioning when the minimizer’s rank $r$ is less than $p$, explaining slower asymptotic convergence observed in practice. Numerical experiments on eigenvalue problems, matrix completion, PhaseLift, and interferometry recovery corroborate these theoretical findings, showing slower convergence for the Burer–Monteiro method in rank-deficient settings and competitive performance for the quotient/embedded approaches. Overall, the work highlights a principled, geometry-driven basis for understanding and selecting among nonconvex PSD fixed-rank solvers.

Abstract

For smooth optimization problems with a Hermitian positive semi-definite fixed-rank constraint, we consider three existing approaches including the simple Burer--Monteiro method, and Riemannian optimization over quotient geometry and the embedded geometry. These three methods can be all represented via quotient geometry with three Riemannian metrics $g^i(\cdot, \cdot)$ $(i=1,2,3)$. By taking the nonlinear conjugate gradient method (CG) as an example, we show that CG in the factor-based Burer--Monteiro approach is equivalent to Riemannian CG on the quotient geometry with the Bures-Wasserstein metric $g^1$. Riemannian CG on the quotient geometry with the metric $g^3$ is equivalent to Riemannian CG on the embedded geometry. For comparing the three approaches, we analyze the condition number of the Riemannian Hessian near the minimizer under the three different metrics. Under certain assumptions, the condition number from the Bures-Wasserstein metric $g^1$ is significantly different from the other two metrics. Numerical experiments show that the Burer--Monteiro CG method has obviously slower asymptotic convergence rate when the minimizer is rank deficient, which is consistent with the condition number analysis.

Figures (9)

• Figure 1: Eigenvalue problem of a random 50 000-by-50 000 PSD matrix of rank 10 solved on the rank 15 manifold: a comparison of normalized cost function value $\frac{\left\lVert Y_kY_k^* - A\right\rVert_F}{\left\lVert A\right\rVert_F}$ decrease versus iteration number $k$ when using L-BFGS approach and CG method with metric $g^i,i=1,2,3$.
• Figure 2: Numerical justification of Theorem \ref{['thm:RQ']} for the eigenvalue problem of a random 50 000-by-50 000 PSD matrix of rank 15 on the rank 15 manifold. Effect of condition number of $A$ on the convergence speed of normalized cost function value $\frac{\left\lVert Y_kY_k^* - A\right\rVert_F}{\left\lVert A\right\rVert_F}$ versus iteration number $k$. (a): when the condition number of $A$ is large, CG with metric $g^1$ is slower; (b): when the condition number of $A$ is smaller, CG with metric $g^1$ becomes faster.
• Figure 3: Numerical justification of Assumption \ref{['assm:gradient_vanish_speed']} for the eigenvalue problem of a random 50 000-by-50 000 PSD matrix of rank 10 on the rank 15 manifold, same setup as the numerical test shown in Fig \ref{['fig:eigenvalue_problem']}. Plots show the ratio term $\frac{\left\lVert\nabla f(Y_k Y_k^*)\right\rVert }{ ({\sigma_p})_k }$ in Assumption \ref{['assm:gradient_vanish_speed']} versus the iteration number $k$ for L-BFGS approach and CG method with metric $g^i,i=1,2,3$.
• Figure 4: Matrix completion of a random 10 000-by-10 000 PSD matrix of rank 25 observed at random 90% entries. A comparison of decrease in normalized cost function value $\frac{\left\lVert P_\Omega(Y_kY_k^* - A)\right\rVert_F}{\left\lVert P_\Omega(A)\right\rVert_F}$ versus iteration number $k$ when using L-BFGS approach and CG method with metric $g^i,i=1,2,3$. When the minimizer is rank deficient (the case in (a)), L-BFGS approach and CG method with metric $g^1$ is significantly slower.
• Figure 5: Numerical justification of Assumption \ref{['assm:gradient_vanish_speed']} for the matrix completion problem of a random 10 000-by-10 000 PSD matrix of rank 25 observed at random 90% entries solved on the rank 30 manifold (same setup as the numerical test shown in Fig \ref{['fig:matrixcompletion-a']}). Plots show the ratio term $\frac{\left\lVert\nabla f(Y_k Y_k^*)\right\rVert }{ ({\sigma_p})_k }$ in the Assumption \ref{['assm:gradient_vanish_speed']} versus the iteration number $k$ for L-BFGS approach and CG method with metric $g^i,i=1,2,3$.

Theorems & Definitions (81)

• Theorem 3.1
• proof
• Theorem 3.2
• Remark 3.3
• proof
• Lemma 3.4
• proof
• Theorem 3.5
• proof
• Remark 3.6