Low-rank Bayesian matrix completion via geodesic Hamiltonian Monte Carlo on Stiefel manifolds

TL;DR

A new sampling-based approach for enabling efficient computation of low-rank Bayesian matrix completion and quantifying the associated uncertainty by designing a geodesic Hamiltonian Monte Carlo (-within-Gibbs) algorithm for generating posterior samples of the SVD factor matrices.

Abstract

We present a new sampling-based approach for enabling efficient computation of low-rank Bayesian matrix completion and quantifying the associated uncertainty. Firstly, we design a new prior model based on the singular-value-decomposition (SVD) parametrization of low-rank matrices. Our prior is analogous to the seminal nuclear-norm regularization used in non-Bayesian setting and enforces orthogonality in the factor matrices by constraining them to Stiefel manifolds. Then, we design a geodesic Hamiltonian Monte Carlo (-within-Gibbs) algorithm for generating posterior samples of the SVD factor matrices. We demonstrate that our approach resolves the sampling difficulties encountered by standard Gibbs samplers for the common two-matrix factorization used in matrix completion. More importantly, the geodesic Hamiltonian sampler allows for sampling in cases with more general likelihoods than the typical Gaussian likelihood and Gaussian prior assumptions adopted in most of the existing Bayesian matrix completion literature. We demonstrate an applications of our approach to fit the categorical data of a mice protein dataset and the MovieLens recommendation problem. Numerical examples demonstrate superior sampling performance, including better mixing and faster convergence to a stationary distribution. Moreover, they demonstrate improved accuracy on the two real-world benchmark problems we considered.

Figures (20)

• Figure 1: Examples of 2D marginals of the posteriors obtained using standard Gaussian priors for the inference of real-valued low-rank matrix factorizations DeGorodetsky2020.
• Figure 2: One random experiment of Case #1 of the synthetic example, with $10\%$ data sampling rate. From left to right: MCMC traces produced by $\mathbf{A}\mathbf{B}^T$ and SVD, and the average autocorrelation times $\pm$ standard deviations produced by HMC. Error bars are obtained using 30 random experiments.
• Figure 3: Case #1 of the synthetic example. Histograms of prediction MADs estimated using different approaches. The reported error bars are obtained using 30 random experiments. The SVD model is sampled by HMC, while the $\mathbf{AB}^T$ model is sampled by Gibbs.
• Figure 4: Histograms of prediction MADs for positive matrices. (a): Case #2. (b): mice data. For both (a) and (b), the top row and bottom row correspond to $10\%$ and $40\%$ data sampling rates, respectively; from the left column to the right column we have the S-SVD, SVD, and $\mathbf{A}\mathbf{B}^T$ models. The S-SVD model generally achieves better performance with smaller tails than the $\mathbf{AB}^T$ model.
• Figure 5: Histograms of prediction MADs for rating data. (a): Case #3, where the top row and bottom row correspond to $10\%$ and $40\%$ data sampling rates, respectively. (b): MovieLens. For both (a) and (b), from the left column to the right column we have the B-SVD, SVD, and $\mathbf{A}\mathbf{B}^T$ models.