Efficient Matrix Factorization Via Householder Reflections

TL;DR

This work studies exact and approximate recovery guarantees for factorizing a data matrix as a Householder reflection times a (binary or Bernoulli) coefficient matrix, i.e., $\mathbf{Y}=\mathbf{H}\mathbf{X}$ with $\mathbf{H}=\mathbf{I}-2\mathbf{u}\mathbf{u}^T$. It proves strong identifiability under a binary $\mathbf{X}$ with only $p=\Omega(1)$ columns (two suffice) and shows that non-binary $\mathbf{X}$ generally precludes unique recovery. Under a Bernoulli model for $\mathbf{X}$, it provides high-probability parameter recovery for $\theta$ and a polynomial-time $(O(np))$ recovery of $\mathbf{u}$ with $\ell_{\infty}$ error decaying as $p$ grows, requiring only $p=\Omega\left(\frac{\log(2n^2)}{8t^2\theta^2 c^2}\right)$. The paper also presents non-iterative algorithms that exploit the Householder structure, along with simulations validating the theoretical guarantees. Together, these results offer a non-iterative pathway toward reliable orthogonal dictionary factorization with strong theoretical guarantees and reduced computational burden.

Abstract

Motivated by orthogonal dictionary learning problems, we propose a novel method for matrix factorization, where the data matrix $\mathbf{Y}$ is a product of a Householder matrix $\mathbf{H}$ and a binary matrix $\mathbf{X}$. First, we show that the exact recovery of the factors $\mathbf{H}$ and $\mathbf{X}$ from $\mathbf{Y}$ is guaranteed with $Ω(1)$ columns in $\mathbf{Y}$ . Next, we show approximate recovery (in the $l\infty$ sense) can be done in polynomial time($O(np)$) with $Ω(\log n)$ columns in $\mathbf{Y}$ . We hope the techniques in this work help in developing alternate algorithms for orthogonal dictionary learning.

Figures (1)

• Figure 1: Infinity norm error for varying number of columns ($\theta=0.1, \theta=0.4$)

Theorems & Definitions (12)

• Theorem 1
• Theorem 2
• Lemma 1
• Theorem 3
• proof
• Lemma 2
• proof
• Lemma 3
• proof
• proof