Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On joint eigen-decomposition of matrices

Erik Troedsson, Daniel Falkowski, Carl-Fredrik Lidgren, Herwig Wendt, Marcus Carlsson

TL;DR

This work addresses the problem of approximately jointly diagonalizing a set of diagonalizable matrices by analyzing the cost functional f_\mathcal{A}(Q) on GL(n,\mathbb{F}). It proves that f_\mathcal{A}(Q) tends to infinity near any rank-deficient point Z whenever the matrices share no common nontrivial invariant subspace, establishing well-posedness with probability one under absolute continuity assumptions, and it derives unified higher-order derivative representations along with explicit gradient and Hessian formulas to enable advanced descent-based optimization. The paper also treats the self-adjoint case, showing how to perform optimization on the unitary group via a Keller projection and detailing how the gradient simplifies when matrices are self-adjoint. Collectively, these results provide rigorous guarantees of well-posedness and furnish the mathematical tools needed to develop robust numerical schemes for joint diagonalization, with direct implications for signal processing and related applications.

Abstract

The problem of approximate joint diagonalization of a collection of matrices arises in a number of diverse engineering and signal processing problems. This problem is usually cast as an optimization problem, and it is the main goal of this publication to provide a theoretical study of the corresponding cost-functional. As our main result, we prove that this functional tends to infinity in the vicinity of rank-deficient matrices with probability one, thereby proving that the optimization problem is well posed. Secondly, we provide unified expressions for its higher-order derivatives in multilinear form, and explicit expressions for the gradient and the Hessian of the functional in standard form, thereby opening for new improved numerical schemes for the solution of the joint diagonalization problem. A special section is devoted to the important case of self-adjoint matrices.

On joint eigen-decomposition of matrices

TL;DR

This work addresses the problem of approximately jointly diagonalizing a set of diagonalizable matrices by analyzing the cost functional f_\mathcal{A}(Q) on GL(n,\mathbb{F}). It proves that f_\mathcal{A}(Q) tends to infinity near any rank-deficient point Z whenever the matrices share no common nontrivial invariant subspace, establishing well-posedness with probability one under absolute continuity assumptions, and it derives unified higher-order derivative representations along with explicit gradient and Hessian formulas to enable advanced descent-based optimization. The paper also treats the self-adjoint case, showing how to perform optimization on the unitary group via a Keller projection and detailing how the gradient simplifies when matrices are self-adjoint. Collectively, these results provide rigorous guarantees of well-posedness and furnish the mathematical tools needed to develop robust numerical schemes for joint diagonalization, with direct implications for signal processing and related applications.

Abstract

The problem of approximate joint diagonalization of a collection of matrices arises in a number of diverse engineering and signal processing problems. This problem is usually cast as an optimization problem, and it is the main goal of this publication to provide a theoretical study of the corresponding cost-functional. As our main result, we prove that this functional tends to infinity in the vicinity of rank-deficient matrices with probability one, thereby proving that the optimization problem is well posed. Secondly, we provide unified expressions for its higher-order derivatives in multilinear form, and explicit expressions for the gradient and the Hessian of the functional in standard form, thereby opening for new improved numerical schemes for the solution of the joint diagonalization problem. A special section is devoted to the important case of self-adjoint matrices.
Paper Structure (7 sections, 11 theorems, 55 equations)

This paper contains 7 sections, 11 theorems, 55 equations.

Table of Contents

  1. Introduction
  2. Behaviour near rank-deficient points
  3. $\mathcal{A}$ has no nontrivial invariant subspace with probability one
  4. Higher order derivatives
  5. The gradient
  6. The Hessian
  7. Remarks on the self-adjoint case

Key Result

Theorem 2.1

We have that $\lim_{Q\to Z}f_{\mathcal{A}}(Q) = \infty$ for all fixed nonzero rank-deficient matrices $Z$, if and only if the matrices in $\mathcal{A}$ have no common nontrivial invariant subspace.

Theorems & Definitions (20)

  • Theorem 2.1
  • Lemma 2.2
  • proof
  • Lemma 2.3
  • proof
  • proof : Proof of Theorem \ref{['diverg']}
  • Proposition 3.1
  • proof
  • Lemma 3.2
  • proof
  • ...and 10 more