Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Characterizing GSVD by singular value expansion of linear operators and its computation

Haibo Li

TL;DR

This work addresses the challenge of computing nontrivial GSVD components for large-scale matrix pairs $\{A,L\}$. It introduces a novel viewpoint that interprets GSVD through the singular value expansion (SVE) of two linear operators $\mathcal{A}$ and $\mathcal{L}$ induced by $\{A,L\}$, with the key role played by the positive semidefinite matrix $M = A^{\top}A + L^{\top}L$. Building on this, the authors develop an operator-based generalization of Golub–Kahan bidiagonalization (gGKB) to approximate the SVEs of $\mathcal{A}$ and $\mathcal{L}$, leading to the gGKB_GSVD algorithm that computes extreme nontrivial GSVD components efficiently for large-scale problems. They establish a residual-based stopping criterion and provide preliminary convergence results under exact arithmetic, illustrating the method's potential via numerical experiments. The approach offers a unifying framework that connects GSVD to Krylov subspace methods for SVD, enabling scalable partial GSVD computations with practical impact in applications requiring relationships between matrix pairs. $M = A^{\top}A + L^{\top}L$ and the operator pair $\mathcal{A},\mathcal{L}$ are central to the theory, with the nontrivial GSVD components corresponding to SVEs of these operators.

Abstract

The generalized singular value decomposition (GSVD) of a matrix pair $\{A, L\}$ with $A\in\mathbb{R}^{m\times n}$ and $L\in\mathbb{R}^{p\times n}$ generalizes the singular value decomposition (SVD) of a single matrix. In this paper, we provide a new understanding of GSVD from the viewpoint of SVD, based on which we propose a new iterative method for computing nontrivial GSVD components of a large-scale matrix pair. By introducing two linear operators $\mathcal{A}$ and $\mathcal{L}$ induced by $\{A, L\}$ between two finite-dimensional Hilbert spaces and applying the theory of singular value expansion (SVE) for linear compact operators, we show that the GSVD of $\{A, L\}$ is nothing but the SVEs of $\mathcal{A}$ and $\mathcal{L}$. This result characterizes completely the structure of GSVD for any matrix pair with the same number of columns. As a direct application of this result, we generalize the standard Golub-Kahan bidiagonalization (GKB) that is a basic routine for large-scale SVD computation such that the resulting generalized GKB (gGKB) process can be used to approximate nontrivial extreme GSVD components of $\{A, L\}$, which is named the gGKB\_GSVD algorithm. We use the GSVD of $\{A, L\}$ to study several basic properties of gGKB and also provide preliminary results about convergence and accuracy of gGKB\_GSVD for GSVD computation. Numerical experiments are presented to demonstrate the effectiveness of this method.

Characterizing GSVD by singular value expansion of linear operators and its computation

TL;DR

This work addresses the challenge of computing nontrivial GSVD components for large-scale matrix pairs . It introduces a novel viewpoint that interprets GSVD through the singular value expansion (SVE) of two linear operators and induced by , with the key role played by the positive semidefinite matrix . Building on this, the authors develop an operator-based generalization of Golub–Kahan bidiagonalization (gGKB) to approximate the SVEs of and , leading to the gGKB_GSVD algorithm that computes extreme nontrivial GSVD components efficiently for large-scale problems. They establish a residual-based stopping criterion and provide preliminary convergence results under exact arithmetic, illustrating the method's potential via numerical experiments. The approach offers a unifying framework that connects GSVD to Krylov subspace methods for SVD, enabling scalable partial GSVD computations with practical impact in applications requiring relationships between matrix pairs. and the operator pair are central to the theory, with the nontrivial GSVD components corresponding to SVEs of these operators.

Abstract

The generalized singular value decomposition (GSVD) of a matrix pair with and generalizes the singular value decomposition (SVD) of a single matrix. In this paper, we provide a new understanding of GSVD from the viewpoint of SVD, based on which we propose a new iterative method for computing nontrivial GSVD components of a large-scale matrix pair. By introducing two linear operators and induced by between two finite-dimensional Hilbert spaces and applying the theory of singular value expansion (SVE) for linear compact operators, we show that the GSVD of is nothing but the SVEs of and . This result characterizes completely the structure of GSVD for any matrix pair with the same number of columns. As a direct application of this result, we generalize the standard Golub-Kahan bidiagonalization (GKB) that is a basic routine for large-scale SVD computation such that the resulting generalized GKB (gGKB) process can be used to approximate nontrivial extreme GSVD components of , which is named the gGKB\_GSVD algorithm. We use the GSVD of to study several basic properties of gGKB and also provide preliminary results about convergence and accuracy of gGKB\_GSVD for GSVD computation. Numerical experiments are presented to demonstrate the effectiveness of this method.
Paper Structure (13 sections, 15 theorems, 54 equations, 5 figures, 2 algorithms)

This paper contains 13 sections, 15 theorems, 54 equations, 5 figures, 2 algorithms.

Table of Contents

  1. Keywords
  2. MSC codes
  3. Introduction
  4. GSVD, SVD and Golub-Kahan bidiagonalization
  5. Characterizing GSVD by singular value expansion of linear operators
  6. Linear operator induced by matrices
  7. Characterizing GSVD by singular value expansion
  8. Generalizing the Golub-Kahan bidiagonalization
  9. GSVD computation by generalized Golub-Kahan bidiagonalization
  10. Computing nontrivial GSVD components by gGKB
  11. Convergence and accuracy
  12. Experimental results
  13. Conclusion and outlook

Key Result

Theorem 1.1

Let $A\in\mathbb{R}^{m\times n}$ and $L\in\mathbb{R}^{p\times n}$ with $\mathrm{rank}((A^{\top},L^{\top})^{\top})=r$. Then the GSVD of $\{A, L\}$ is where $q_1+q_2+q_3=r$, and $P_{A}\in \mathbb{R}^{m\times m}$, $P_{L}\in \mathbb{R}^{p\times p}$ are orthogonal, $X\in\mathbb{R}^{n\times n}$ is invertible, and $\Sigma_{A}^{T}\Sigma_A+\Sigma_{L}^{T}\Sigma_L=I_{r}$. The values of $q_1$, $q_2$ and $q_3

Figures (5)

  • Figure 6.1: Convergence and accuracy of approximations to $c_i$. Top: convergence of Ritz values $\theta_{i}^{(k)}$ to largest/smallest $c_i$ by gGKB_GSVD without reorthogonalization (left) and with full reorthogonalization (right). Bottom: error curves (full reorthogonalization).
  • Figure 6.2: Convergence and accuracy of approximations to $s_i$. Top: convergence of Ritz values $\theta_{i}^{(k)}$ to largest/smallest $s_i$ by gGKB_GSVD without reorthogonalization (left) and with full reorthogonalization (right). Bottom: error curves (full reorthogonalization).
  • Figure 6.3: Error curves of the approximate GSVD components by gGKB_GSVD and relative residual norm with its upper bound. Left: approximations to the 1-st GSVD components. Right: approximations to the n-th GSVD components.
  • Figure 6.4: Error curves of the approximate GSVD components by gGKB_GSVD and relative residual norm with its upper bound, where $\mathrm{rank}((A^{\top}, L^{\top})^\top) = r<n$. Left: approximations to the 1-st GSVD components. Right: approximations to the r-th GSVD components.
  • Figure 6.5: Accuracy of computed GSVD components by gGKB_GSVD, where $s=M^{\dag}\bar{s}$ at each gGKB iteration is computed by solving $\min_{s}\|Ms-\bar{s}\|_2$ using lsqr.m with different stopping tolerance tol. Top: tol=$10^{-10}$. Top: tol=$10^{-8}$.

Theorems & Definitions (16)

  • Theorem 1.1: GSVD
  • Proposition 2.1
  • Lemma 3.1
  • Lemma 3.2
  • Lemma 3.3
  • Theorem 3.1
  • Theorem 3.2
  • Theorem 3.3
  • Remark 4.1
  • Proposition 4.1
  • ...and 6 more