A randomized algorithm for simultaneously diagonalizing symmetric matrices by congruence
Haoze He, Daniel Kressner
TL;DR
The paper addresses the problem of simultaneously diagonalizing a family of real symmetric matrices by congruence (SDC) under noise. It introduces a randomized approach (RSDC) that reduces SDC to a generalized eigenvalue problem by formulating two random linear combinations of the matrix family, and provides exact recovery guarantees for exactly SDC families as well as robust guarantees under mild regularity and PD assumptions. The authors further enhance practicality by combining RSDC with an optimization-based refinement (RFFDIAG), achieving substantial efficiency gains while maintaining accuracy across synthetic data, blind source separation, image separation, and EEG tasks. These results demonstrate that RSDC-based methods offer a fast, reliable alternative to purely optimization-based solvers, with provable recovery properties and strong empirical performance. The work has direct impact on signal processing applications such as BSS and tensor CP decompositions, where stable, scalable SDC is essential.
Abstract
A family of symmetric matrices $A_1,\ldots, A_d$ is SDC (simultaneous diagonalization by congruence, also called non-orthogonal joint diagonalization) if there is an invertible matrix $X$ such that every $X^T A_k X$ is diagonal. In this work, a novel randomized SDC (RSDC) algorithm is proposed that reduces SDC to a generalized eigenvalue problem by considering two (random) linear combinations of the family. We establish exact recovery: RSDC achieves diagonalization with probability $1$ if the family is exactly SDC. Under a mild regularity assumption, robust recovery is also established: Given a family that is $ε$-close to SDC then RSDC diagonalizes, with high probability, the family up to an error of norm $\mathcal{O}(ε)$. Under a positive definiteness assumption, which often holds in applications, stronger results are established, including a bound on the condition number of the transformation matrix. For practical use, we suggest to combine RSDC with an optimization algorithm. The performance of the resulting method is verified for synthetic data, image separation and EEG analysis tasks. It turns out that our newly developed method outperforms existing optimization-based methods in terms of efficiency while achieving a comparable level of accuracy.