Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A unifying framework for ADI-like methods for linear matrix equations and beneficial consequences

Jonas Schulze, Jens Saak

TL;DR

The paper presents a unifying ADI framework based on commuting operator splits and extends low-rank ADI methods to arbitrary (nonzero) initial values for linear matrix equations, including Lyapunov and Sylvester forms. It develops fully commuting splitting schemes, proving permutation invariance of iterates and real-valued double-steps for complex shifts, and applies the approach to both algebraic and differential Riccati settings using outer Newton and Rosenbrock methods. Numerical experiments show that nonzero initial ADI values can significantly reduce the total number of ADI steps and, in some configurations, achieve substantial speed-ups (up to 8x in the DRE case and up to 17% in the ARE case), though performance depends on shift strategies and problem structure. The framework broadens the applicability and efficiency of ADI-type methods for large-scale linear matrix equations and provides practical guidance for shift selection and outer-iteration coupling.

Abstract

We derive the alternating-directions implicit (ADI) method based on a commuting operator split and apply the results in detail to the continuous time algebraic Lyapunov equation with low-rank constant term and approximate solution, giving pointers for the Sylvester case. Previously, it has been mandatory to start the low-rank ADI for Lyapunov equations (CF-ADI, LR-ADI, G-LR-ADI) or Sylvester equations (fADI, G-fADI) with an all-zero initial value. Our approach extends the known efficient iteration schemes of low-rank increments and residuals to arbitrary low-rank initial values for all these methods. We further generalize two properties of the low-rank Lyapunov ADI to the generic ADI applied to arbitrary linear equations using a commuting operator split, namely the invariance of iterates under permutations of the shift parameters, and the efficient handling of complex shift parameters. We investigate the performance of arbitrary initial values using two outer iterations in which the low-rank Lyapunov ADI is typically called. First, we solve an algebraic Riccati equation with the Newton method. Second, we solve a differential Riccati equation with a first-order Rosenbrock method. Numerical experiments confirm that the proposed new initial value of the ADI can lead to a significant reduction in the total number of ADI steps, while also showing a 17% and 8x speed-up over the zero initial value for the two equation types, respectively.

A unifying framework for ADI-like methods for linear matrix equations and beneficial consequences

TL;DR

The paper presents a unifying ADI framework based on commuting operator splits and extends low-rank ADI methods to arbitrary (nonzero) initial values for linear matrix equations, including Lyapunov and Sylvester forms. It develops fully commuting splitting schemes, proving permutation invariance of iterates and real-valued double-steps for complex shifts, and applies the approach to both algebraic and differential Riccati settings using outer Newton and Rosenbrock methods. Numerical experiments show that nonzero initial ADI values can significantly reduce the total number of ADI steps and, in some configurations, achieve substantial speed-ups (up to 8x in the DRE case and up to 17% in the ARE case), though performance depends on shift strategies and problem structure. The framework broadens the applicability and efficiency of ADI-type methods for large-scale linear matrix equations and provides practical guidance for shift selection and outer-iteration coupling.

Abstract

We derive the alternating-directions implicit (ADI) method based on a commuting operator split and apply the results in detail to the continuous time algebraic Lyapunov equation with low-rank constant term and approximate solution, giving pointers for the Sylvester case. Previously, it has been mandatory to start the low-rank ADI for Lyapunov equations (CF-ADI, LR-ADI, G-LR-ADI) or Sylvester equations (fADI, G-fADI) with an all-zero initial value. Our approach extends the known efficient iteration schemes of low-rank increments and residuals to arbitrary low-rank initial values for all these methods. We further generalize two properties of the low-rank Lyapunov ADI to the generic ADI applied to arbitrary linear equations using a commuting operator split, namely the invariance of iterates under permutations of the shift parameters, and the efficient handling of complex shift parameters. We investigate the performance of arbitrary initial values using two outer iterations in which the low-rank Lyapunov ADI is typically called. First, we solve an algebraic Riccati equation with the Newton method. Second, we solve a differential Riccati equation with a first-order Rosenbrock method. Numerical experiments confirm that the proposed new initial value of the ADI can lead to a significant reduction in the total number of ADI steps, while also showing a 17% and 8x speed-up over the zero initial value for the two equation types, respectively.
Paper Structure (11 sections, 7 theorems, 55 equations, 4 figures, 3 algorithms)

This paper contains 11 sections, 7 theorems, 55 equations, 4 figures, 3 algorithms.

Table of Contents

  1. Intro
  2. Nonstationary Splitting Schemes
  3. Commuting Splitting Schemes
  4. Fully Commuting Splitting Schemes
  5. ADI Method
  6. Low-rank ADI
  7. Applications and Numerical Experiments
  8. Algebraic Equation
  9. Differential Equation
  10. Conclusion
  11. Order of ADI shifts

Key Result

Proposition 2.1

Let $A = M_{k} - N_{k}$ define a commuting nonstationary splitting method eq:normalform. Then, the residual $r^k := Ax^k - b$ adheres to where $G_{k} := M_{k}^{-1} N_{k}$ denotes the iteration matrix.

Figures (4)

  • Figure 1: Newton method (hybrid w/ line search) applied to solve ARE \ref{['eq:are']} arising from \ref{['ex:are']}. ADI shifts: $\heuristic(10, 10, 10)$. The ADI is started from a zero value (\ref{['fig:are:macro:initzero']}) or with the previous Newton iterate (\ref{['fig:are:macro:initprev']}).
  • Figure 2: Normalized residuals $\norm{\Ricc(X_{\ell+1,k})} / \norm{\Ricc(0)}$ following the naive formula \ref{['eq:are:naive residual']}, and normalized residuals $\norm{\Lyap_\ell(X_{\ell+1,k})} / \norm{\Lyap_\ell(0)}$ following \ref{['alg:lyapunov']}, over the course of all ADI iterations $k$ during all Newton steps $\ell$. Newton method: hybrid w/ line search. ADI shifts: $\heuristic(10, 10, 10)$. The ADI is started from a zero value (old) or with the previous Newton iterate (new).
  • Figure 3: Rosenbrock method applied to solve DRE \ref{['eq:dre']} arising from \ref{['ex:dre']}. ADI shifts: $\heuristic(20, 30, 30)$. Rosenbrock step size $\tau=10$. The ADI is started from a zero value (\ref{['fig:dre:initzero']}) or with the previous Rosenbrock iterate (\ref{['fig:dre:initprev']}).
  • Figure 4: Spectral radius of Cayley transformations associated to spectrum of $A$, and upper bound $\hat{\rho}_k$ on the norm of parts of the iteration map over the course of multiple ADI iterations $k$, for different permutations of the spectrum $\Lambda(A)$.

Theorems & Definitions (20)

  • Proposition 2.1: Residual Recursion Sch22
  • proof
  • Corollary 2.2
  • Corollary 2.3
  • Proposition 2.4: Permutation Invariance Sch22
  • proof
  • Theorem 3.1: Fully Commuting Splitting Scheme
  • Theorem 3.2: Real-valued Double-Step
  • proof
  • Remark 4.1: Generalized Matrix Equations
  • ...and 10 more