Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Computing Kurdyka-Łojasiewicz exponents via composition and symmetry

Cédric Josz, Wenqing Ouyang

TL;DR

Calculus rules for the Kurdyka-\L{}ojasiewicz (KL) exponent are devised using the rank theorem and Lie group actions, providing a unified framework for establishing linear convergence of various algorithms in matrix factorization, matrix factorization, matrix sensing, and linear neural networks.

Abstract

We devise calculus rules for the Kurdyka-Łojasiewicz (KL) exponent using the rank theorem and Lie group actions. They apply to a wide class of composite and invariant functions, and are particularly suitable for handling nonisolated local minima. Notably, smoothness plays no role, eschewing gradient and Hessian computations. This provides a unified framework for establishing linear convergence of various algorithms in matrix factorization, $\ell_1$-matrix factorization, matrix sensing, and linear neural networks.

Computing Kurdyka-Łojasiewicz exponents via composition and symmetry

TL;DR

Calculus rules for the Kurdyka-\L{}ojasiewicz (KL) exponent are devised using the rank theorem and Lie group actions, providing a unified framework for establishing linear convergence of various algorithms in matrix factorization, matrix factorization, matrix sensing, and linear neural networks.

Abstract

We devise calculus rules for the Kurdyka-Łojasiewicz (KL) exponent using the rank theorem and Lie group actions. They apply to a wide class of composite and invariant functions, and are particularly suitable for handling nonisolated local minima. Notably, smoothness plays no role, eschewing gradient and Hessian computations. This provides a unified framework for establishing linear convergence of various algorithms in matrix factorization, -matrix factorization, matrix sensing, and linear neural networks.
Paper Structure (24 sections, 36 theorems, 282 equations, 1 table)

This paper contains 24 sections, 36 theorems, 282 equations, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Variational analysis
  4. Subanalytic geometry
  5. Differential geometry
  6. Calculus rules
  7. Composition rule
  8. Symmetry rule
  9. Applications
  10. Matrix factorization
  11. Underparametrized case
  12. Linear convergence
  13. Linear neural network
  14. Structure of $XY=M$
  15. Matrix sensing
  16. ...and 9 more sections

Key Result

Theorem 3.2

(Lyusternik-Graves theorem dontchev2021lectures). If $F:\mathbb{R}^n\to\mathbb{R}^m$ is $C^ 1$ near $\overline{x}\in\mathbb{R}^ n$ and a submersion at $\overline{x}$, then $F$ is metrically regular at $\overline{x}$ for $F(\overline{x})$.

Theorems & Definitions (74)

  • proof
  • Definition 2.3
  • Definition 2.4
  • proof
  • Definition 3.1: dontchev2009implicit
  • Theorem 3.2
  • Lemma 3.3
  • Proposition 3.4
  • proof
  • Lemma 3.5
  • ...and 64 more