A space-decoupling framework for optimization on bounded-rank matrices with orthogonally invariant constraints

TL;DR

This work addresses optimization on matrices with a bounded rank under an orthogonally invariant constraint $h$, which yields a coupled, nonsmooth feasible set. The authors develop a space-decoupling framework that reforms the problem into a smooth Riemannian optimization on a manifold $\mathcal{M}_h$, with a mapping $\phi$ linking back to the original problem, and provide a tangent-cone intersection analysis to justify this reformulation. They further derive the Riemannian gradient and Hessian, design first- and second-order retractions, and construct vector transports on $\mathcal{M}_h$, establishing convergence guarantees and equivalence between the reformulated and original problems. The approach is validated through diverse real-world applications—ranging from spherical data fitting and graph similarity to low-rank SDP, Markov-process model reduction, reinforcement learning, and neural-network training—demonstrating improved stability, efficiency, and scalability over traditional methods. Overall, the space-decoupling framework offers a principled, geometry-driven path to solving complex constrained low-rank problems with broad practical impact.

Abstract

Imposing additional constraints on low-rank optimization has garnered growing interest. However, the geometry of coupled constraints hampers the well-developed low-rank structure and makes the problem intricate. To this end, we propose a space-decoupling framework for optimization on bounded-rank matrices with orthogonally invariant constraints. The "space-decoupling" is reflected in several ways. We show that the tangent cone of coupled constraints is the intersection of tangent cones of each constraint. Moreover, we decouple the intertwined bounded-rank and orthogonally invariant constraints into two spaces, leading to optimization on a smooth manifold. Implementing Riemannian algorithms on this manifold is painless as long as the geometry of additional constraints is known. In addition, we unveil the equivalence between the reformulated problem and the original problem. Numerical experiments on real-world applications -- spherical data fitting, graph similarity measuring, low-rank SDP, model reduction of Markov processes, reinforcement learning, and deep learning -- validate the superiority of the proposed framework.

Figures (12)

• Figure 1: Illustration of $\mathbb{R}_{\leq r}^{m \times n}\cap\mathrm{Ob}(m,n)$ with $(m,n,r)=(3,4,2)$. Given a matrix $X\in\mathrm{Ob}(3,4)$, if the first two rows in the unit sphere span a plane, then the rank constraint $\mathop{\mathrm{rank}}\nolimits(X)\le 2$ confines the third row in the intersection of this plane with the sphere.
• Figure 2: Illustration of optimization via the space-decoupling parameterization.
• Figure 3: Relationship among the manifolds and the embedded spaces, where $\pi$ is the quotient mapping induced by the group action of $\mathcal{O}(n-r)$.
• Figure 4: Spherical data fitting problem with the oversampling factor $\mathrm{OS}=5$ and the unbiased rank parameter $r=r^*=6$.
• Figure 5: Graph similarity measuring problem when the solution is low-rank. Left: test with the unbiased rank parameter $r=r^*=1$; Right: test with over-estimated rank parameters $r=10,50,100>r^*$.

Theorems & Definitions (66)

• proposition thmcounterproposition
• proof
• corollary thmcountercorollary
• proof
• lemma thmcounterlemma
• proof
• lemma thmcounterlemma
• proof
• proposition thmcounterproposition
• proof