Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Randomized Joint Diagonalization of Symmetric Matrices

Haoze He, Daniel Kressner

TL;DR

A main novel contribution is to prove robust recovery: Given a family that is $\epsilon$-near to a commuting family, RJD jointly diagonalizes this family, with high probability, up to an error of norm O($\ep silon$).

Abstract

Given a family of nearly commuting symmetric matrices, we consider the task of computing an orthogonal matrix that nearly diagonalizes every matrix in the family. In this paper, we propose and analyze randomized joint diagonalization (RJD) for performing this task. RJD applies a standard eigenvalue solver to random linear combinations of the matrices. Unlike existing optimization-based methods, RJD is simple to implement and leverages existing high-quality linear algebra software packages. Our main novel contribution is to prove robust recovery: Given a family that is $ε$-near to a commuting family, RJD jointly diagonalizes this family, with high probability, up to an error of norm O($ε$). We also discuss how the algorithm can be further improved by deflation techniques and demonstrate its state-of-the-art performance by numerical experiments with synthetic and real-world data.

Randomized Joint Diagonalization of Symmetric Matrices

TL;DR

A main novel contribution is to prove robust recovery: Given a family that is -near to a commuting family, RJD jointly diagonalizes this family, with high probability, up to an error of norm O().

Abstract

Given a family of nearly commuting symmetric matrices, we consider the task of computing an orthogonal matrix that nearly diagonalizes every matrix in the family. In this paper, we propose and analyze randomized joint diagonalization (RJD) for performing this task. RJD applies a standard eigenvalue solver to random linear combinations of the matrices. Unlike existing optimization-based methods, RJD is simple to implement and leverages existing high-quality linear algebra software packages. Our main novel contribution is to prove robust recovery: Given a family that is -near to a commuting family, RJD jointly diagonalizes this family, with high probability, up to an error of norm O(). We also discuss how the algorithm can be further improved by deflation techniques and demonstrate its state-of-the-art performance by numerical experiments with synthetic and real-world data.
Paper Structure (18 sections, 7 theorems, 73 equations, 6 figures, 6 tables, 2 algorithms)

This paper contains 18 sections, 7 theorems, 73 equations, 6 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Nearly commuting vs almost commuting
  3. Common eigenvectors and eigenvalue vectors
  4. Basic RJD algorithm
  5. The algorithm
  6. Correctness for exactly commuting matrices
  7. Analysis of RJD
  8. Preliminary results
  9. Probabilistic bound with gap
  10. Probabilistic bound without gap
  11. Observed failure probability of RJD
  12. A deflation-based RJD algorithm
  13. Numerical experiments
  14. Synthetic data
  15. Blind source separation
  16. ...and 3 more sections

Key Result

Lemma 1

With the notation introduced above, let $\lambda(\mu) = \langle \mu, \Lambda \rangle$ and $\tilde{\lambda}(\mu) = \langle \mu, \tilde{\Lambda} \rangle$ be eigenvalues of $A(\mu)$ for $\mu \sim \mathcal{N}(0, I_d)$. Then $\lambda(\mu) = \tilde{\lambda}(\mu)$ implies $\Lambda = \tilde{\Lambda}$ with p

Figures (6)

  • Figure 1: Empirical failure probability vs. $R - 1$ on log-log scale.
  • Figure 2: Original signals before mixture, the first three signals are the audio sources and the rest is white noise. All signals are sorted according to the energy.
  • Figure 3: Signals after mixture
  • Figure 4: Signals recovered by RJD, signals are sorted according to the energy. The first three signals correspond to the original audio sources before mixture in Figure \ref{['fig:original_signal']}.
  • Figure 5: Signals Recovered by DRJD, signals are sorted according to the energy. The first three signals correspond to the original audio sources before mixture in Figure \ref{['fig:original_signal']}.
  • ...and 1 more figures

Theorems & Definitions (15)

  • Lemma 1
  • proof
  • Theorem 2
  • proof
  • Lemma 3
  • proof
  • Lemma 4
  • proof
  • Corollary 5
  • proof
  • ...and 5 more