Disjunctive Branch-And-Bound for Certifiably Optimal Low-Rank Matrix Completion

TL;DR

The paper develops a certifiably optimal approach to low-rank matrix completion by reformulating the problem with projection matrices and solving via a disjunctive branch-and-bound framework. Central innovations include eigenvector-based disjunctive cuts that tighten a matrix perspective relaxation, a determinant-minor–based convex relaxation that strengthens bounds, and a custom algorithm that interleaves SDP relaxations with alternating-minimization incumbents to efficiently reach near-optimal or optimal solutions on large-scale instances. The method scales to matrices with max{m,n} up to 2500 and rank k up to 5, achieving substantial gains in training and test performance over heuristic approaches. The combination of stronger relaxations, principled disjunctions, and strategic incumbent generation yields certifiable optima for medium-to-large problems and demonstrates practical improvements in reconstruction error on real-like data regimes.

Abstract

Low-rank matrix completion consists of computing a matrix of minimal complexity that recovers a given set of observations as accurately as possible. Unfortunately, existing methods for matrix completion are heuristics that, while highly scalable and often identifying high-quality solutions, do not possess any optimality guarantees. We reexamine matrix completion with an optimality-oriented eye. We reformulate low-rank matrix completion problems as convex problems over the non-convex set of projection matrices and implement a disjunctive branch-and-bound scheme that solves them to certifiable optimality. Further, we derive a novel and often near-exact class of convex relaxations by decomposing a low-rank matrix as a sum of rank-one matrices and incentivizing that two-by-two minors in each rank-one matrix have determinant zero. In numerical experiments, our new convex relaxations decrease the optimality gap by two orders of magnitude compared to existing attempts, and our disjunctive branch-and-bound scheme solves $n \times m$ rank-$r$ matrix completion problems to certifiable optimality or near optimality in hours for $\max \{m, n\} \leq 2500$ and $r \leq 5$. Moreover, this improvement in the training error translates into an average $2\%$--$50\%$ improvement in the test set error.

Figures (9)

• Figure 1: Root node relative gap (a) and time taken (b) at the root node for rank-$1$$n$-by-$n$ matrix completion problems with $2kn \log_{10}(n)$ filled entries, in a regime with low regularization ($\gamma = 80.0$).
• Figure 2: (Scaled) lower bounds of $\mathcal{M}_4$ relaxation relative to the relaxation with no Shor LMIs, with $C n$ or $C n^2$ entries, with low regularization ($\gamma = 80.0$).
• Figure 3: Comparison of time taken to optimality (relative gap $10^{-4}$) for rank-one matrix completion problems with $pn \log_{10}(n)$ filled entries, over different numbers of pieces $q \in \{2, 3, 4\}$ in upper-approximation. We set $p=2.0$ (less entries) in the top plots, and $p=3.0$ (more entries) in the bottom plots. We set $\gamma=20.0$ (more regularization) in the left plots, and $\gamma=80.0$ (less regularization) in the right plots.
• Figure 4: Comparison of relative optimality gap at root node (left) and after running branch-and-bound for three hours (right) for rank-$k$ matrix completion problems of dimension $50 \times m$, with $k m \log_{10}(m)$ filled entries, varying $m$ and $k$, with $\gamma = 120.0$, averaged over 10 random instances.
• Figure 5: Percentage improvement in out-of-sample MSE for rank-$k$$50 \times m$ matrix completion problems, with $k m \log_{10}(m)$ filled entries, varying $m$ and $k$, with $\gamma = 120.0$, averaged over 10 random instances.

Theorems & Definitions (7)

• Proposition 1
• Proposition 2
• Remark 1
• Remark 2
• Lemma 1
• Proposition 3
• Theorem 1