The Pseudoinverse of $A=CR$ is $A^+=R^+C^+$ (?)

Michał P. Karpowicz; Gilbert Strang

The Pseudoinverse of $A=CR$ is $A^+=R^+C^+$ (?)

Michał P. Karpowicz, Gilbert Strang

Abstract

This paper gives three formulas for the pseudoinverse of a matrix product $A = CR$. The first is sometimes correct, the second is always correct, and the third is almost never correct. But that third randomized pseudoinverse $A^+_r$ may be very useful when $A$ is a very large matrix. 1. $A^+ = R^+C^+$ when $A = CR$ and $C$ has independent columns and $R$ has independent rows. 2. $A^+ = (C^+CR)^+(CRR^+)^+$ is always correct. 3. $A^+_r = (P^TCR)^+P^TCRQ(CRQ)^+ = A^+$ only when $\mathrm{rank}(P^TA) = \mathrm{rank}(AQ) = \mathrm{rank}(A)$ with $A = CR$.

The Pseudoinverse of $A=CR$ is $A^+=R^+C^+$ (?)

Abstract

This paper gives three formulas for the pseudoinverse of a matrix product

. The first is sometimes correct, the second is always correct, and the third is almost never correct. But that third randomized pseudoinverse

may be very useful when

is a very large matrix. 1.

when

and

has independent columns and

has independent rows. 2.

is always correct. 3.

only when

with

Paper Structure (4 theorems, 30 equations, 5 figures)

This paper contains 4 theorems, 30 equations, 5 figures.

Key Result

Theorem 1

The pseudoinverse $A^+$ of a product $A=CR$ is the product of the pseudoinverses $R^+C^+$ (as for inverses) when $C$ has full column rank and $R$ has full row rank $r$.

Figures (5)

Figure 1: $A\boldsymbol{x}_\text{\footnotesize ${\boldsymbol{r}}$}=\boldsymbol{b}$ is in the column space of $A$ and $A\boldsymbol{x}_\text{\footnotesize ${\boldsymbol{n}}$}=\boldsymbol{0}$. The complete solution to $A\boldsymbol{x}=\boldsymbol{b}$ is $\boldsymbol{x}=$one $\boldsymbol{x}_\text{\footnotesize ${\boldsymbol{r}}$}\,+$any $\boldsymbol{x}_\text{\footnotesize ${\boldsymbol{n}}$}$.
Figure 2: The four subspaces for $A^{\pmb{+}}$ are the four subspaces for $A^\mathrm{T}$.
Figure 3: Given a full rank decomposition $A=CR$, matrix $R$ maps $\hbox{C}(A^T)$ into $\hbox{C}(C^T)$ and matrix $C^T$ maps $\hbox{C}(A)$ into $\hbox{C}(R)$ when $(CR)^+=R^+C^+$.
Figure 4: The pseudoinverse $A^+=(CR)^+=(C^+CR)^+(CRR^+)^+$ decomposes into the product of the pseudoinverse of $R$ projected on the row space of $C^T$ and the pseudoinverse of $C^T$ projected on the column space of $R$.
Figure 5: Computation time and relative approximation error of the randomized $A^+_r = \mathtt{rpinv}(A)$, randomized SVD-based $A^+_s = \mathtt{rsvdi}(A)$ and direct $A^+ = \mathtt{pinv}(A)$ method of calculating the pseudoinverse of $A$. In the experiment, we generate a random test matrix $A = \mathtt{gallery('randsvd',n,1e100)}$ with $n$ ranging from $100$ to $1000$. Then, we calculate the approximation $A^+_r$ taking random $m$ by $p$ matrix $P$ and $n$ by $q$ matrix $Q$ (in Theorem \ref{['thm3']}) with $p = q =\lceil \alpha \cdot n\rceil$ for $\alpha = 0.4$ (top) and $\alpha = 0.1$ (bottom). Randomized SVD-based pseudoinverse is calculated based on rank $s = \mathrm{rank}(A)$ approximation $A_s = QU_s\Sigma_sV_s^T$ for $[Q,\sim]=\mathrm{qr}(A*\mathtt{randn}(n,s))$.

Theorems & Definitions (4)

Theorem 1
Lemma 1
Theorem 2
Theorem 3