The Relation Between the EVD or SVD of Summands and the EVD or SVD of the Sum

TL;DR

The paper addresses how the eigenvalue decomposition (EVD) and singular value decomposition (SVD) of a sum relate to the EVD/SVD of the summands. It introduces a block-inner-product construction $C = Z^T Z$ with $Z = [\mathbf{PD} \ \mathbf{QE}]$, where $A = \mathbf{P}\mathbf{D}^2\mathbf{P}^T$ and $B = \mathbf{Q}\mathbf{E}^2\mathbf{Q}^T$, and shows that the spectrum of $C$ can be obtained by projecting eigenvectors from $Z^T Z$ onto $Z$. The method is extended to the SVD of ${\mathbf{X}}^T{\mathbf{Y}}$ via EVD of a sum of PSD matrices, using a traceless augmented form and an eigenvalue shift, and generalized to the SVD of a sum ${\mathbf{F}}+{\mathbf{G}}$. The results suggest practical computational gains for data matrices with few rows and many columns and have potential applications in multi-omics and generalized Procrustes analysis.

Abstract

In this work, we show how the eigenstructures of summands are related to that of the sum. In particular, we show that the sum of two positive semidefinite matrices can be written as the inner product of two block matrices $\mathbf{C} = \mathbf{A} + \mathbf{B} = \mathbf{PD}^2\mathbf{P}^T + \mathbf{QE}^2\mathbf{Q}^T = \begin{pmatrix} \mathbf{PD} & \mathbf{QE} \end{pmatrix}\begin{pmatrix} \mathbf{PD} & \mathbf{QE} \end{pmatrix}^T = \mathbf{Z}^T\mathbf{Z}$, such that the eigenvector decomposition of $\mathbf{C}$ can be obtained by projecting the eigenvectors from the block matrix product $\mathbf{ZZ}^T$ onto the block matrix $\mathbf{Z}^T = \begin{pmatrix} \mathbf{PD} & \mathbf{QE} \end{pmatrix}$. Next, it is shown that the result can be used to rewrite the SVD of a matrix inner product $\mathbf{X}^T\mathbf{Y}$, utilizing the eigenstructures of $\mathbf{X}$ and $\mathbf{Y}$. Finally, it is shown how this can be generalized to express the SVD of a sum of arbitrary matrices $\mathbf{H} = \mathbf{F} + \mathbf{G}$ in terms of the SVDs of the summands. The results may be useful in algorithms for eigendecomposition, specifically if the eigenproblem can be expressed in terms of a sum of matrices with simple eigenstructures as, e.g., in multi-omics research and generalized Procrustes analysis.