Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Linear Algebra Problems Solved by Using Damped Dynamical Systems on the Stiefel Manifold

M Gulliksson, A Oleynik, M Ogren, R Bakhshandeh-Chamazkoti

TL;DR

This work develops two damped dynamical systems for minimizing functions on the Stiefel manifold: a Lagrange‑function approach with an auxiliary constraint dynamics and a projected‑gradient approach that stays on the manifold via projection. Both yield stationary points satisfying the KKT conditions and are shown to be asymptotically stable through linearization analyses and Sylvester‑type equations that determine the Lagrange parameters. The framework is demonstrated on canonical problems—the linear eigenvalue problem and the orthogonal Procrustes problem—deriving explicit Jacobian structures and stability criteria. Numerical experiments using a symplectic Euler scheme illustrate convergence behavior, compare the two formulations, and highlight how problem structure and damping parameters influence performance. The results point to practical avenues for parameter selection and motivate extending the approach to broader Riemannian manifolds with energy‑preserving integrators.

Abstract

We develop a new method for solving minimization problems on the Stiefel Manifold using damped dynamical systems. The constraints are satisfied in the limit by an additional damped dynamical system. The method is illustrated by numerical experiments and compared to a state-of-the-art conjugate gradient method.

Linear Algebra Problems Solved by Using Damped Dynamical Systems on the Stiefel Manifold

TL;DR

This work develops two damped dynamical systems for minimizing functions on the Stiefel manifold: a Lagrange‑function approach with an auxiliary constraint dynamics and a projected‑gradient approach that stays on the manifold via projection. Both yield stationary points satisfying the KKT conditions and are shown to be asymptotically stable through linearization analyses and Sylvester‑type equations that determine the Lagrange parameters. The framework is demonstrated on canonical problems—the linear eigenvalue problem and the orthogonal Procrustes problem—deriving explicit Jacobian structures and stability criteria. Numerical experiments using a symplectic Euler scheme illustrate convergence behavior, compare the two formulations, and highlight how problem structure and damping parameters influence performance. The results point to practical avenues for parameter selection and motivate extending the approach to broader Riemannian manifolds with energy‑preserving integrators.

Abstract

We develop a new method for solving minimization problems on the Stiefel Manifold using damped dynamical systems. The constraints are satisfied in the limit by an additional damped dynamical system. The method is illustrated by numerical experiments and compared to a state-of-the-art conjugate gradient method.

Paper Structure

This paper contains 25 sections, 14 theorems, 157 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Optimizing on the Stiefel manifold
  3. Damped Dynamical Systems
  4. Lagrange Function Formulation
  5. Projected Gradient Formulation
  6. The linear eigenvalue problem
  7. The orthogonal Procrustes Problem
  8. Asymptotic Stability
  9. First-order Systems
  10. Eigenvalue Structure for the Lagrange Approach
  11. Eigenvalue Structure when using Projection
  12. The Eigenvalue Problem for the Lagrange formulation
  13. Eigenvalues of the Jacobian and Asymptotic Stability
  14. The Projected Gradient System for the Eigenvalue Problem
  15. The Procrustes Problem using the Lagrange Formulation
  16. ...and 10 more sections

Key Result

Lemma 2.1

Let $\mathbf{X}\in{\mathbb R} ^{n\times p}$ be orthonormal and $\mathbf{G}\in{\mathbb R}^{n\times p}$ where then $(\mathbf{I} - \mathbf{X} \mathbf{X}^\top)\mathbf{G} = 0$ and $\mathbf{X}^\top \mathbf{G} - \mathbf{G}^\top \mathbf{X} = 0$.

Figures (7)

  • Figure 1: Eigenvalue problem. Convergence of $\mathbf{Y}_k$ (see main text) vs iteration steps $k$. Blue curves are for the Lagrange approach and red for the projection. 'Right-hand-side' is referring to the norm of the right-hand-side in the second-order damped system, respectively. The problem size was $n=100, p=10$ and the initial $\mathbf{Y}_{k=0}$ was chosen as a random matrix with elements uniformly distributed in $\left[-100,100 \right]$.
  • Figure 2: Procrustes problem. Convergence of $\mathbf{Y}_k$ (see main text) vs iteration steps $k$. The parameters and initial conditions are the same as in Figure \ref{['fig:EFeig']}. Note in the upper-right subfigure that the Lagrange approach finds a lower local minimum.
  • Figure 3: Eigenvalue problem. Convergence of $\| \mathbf{C} \|_F$ vs $\| \mathbf{A} \mathbf{X} + \mathbf{X} \mathbf{M} \|_F$.
  • Figure 4: Procrustes problem. Convergence of $\| \mathbf{C} \|_F$ vs $\| \mathbf{A} \mathbf{X} + \mathbf{X} \mathbf{M} \|_F$.
  • Figure 5: A large $\nu$ gives fast convergence towards the manifold.
  • ...and 2 more figures

Theorems & Definitions (24)

  • Lemma 2.1
  • proof
  • Definition 6.1: Asymptotic Stability
  • Theorem 6.1: Asymptotic Stability for Nonlinear Systems
  • proof
  • Theorem 6.2: Asymptotic Stability for Constrained Systems
  • proof
  • Lemma 8.1
  • proof
  • Lemma 8.2
  • ...and 14 more