Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Stochastic Gradient Descents on Manifolds with an Application on Weighted Low-Rank Approximation

Peiqi Yang, Conglong Xu, Hao Wu

Abstract

We prove a convergence theorem for stochastic gradient descents on manifolds with adaptive learning rate and apply it to the weighted low-rank approximation problem.

Adaptive Stochastic Gradient Descents on Manifolds with an Application on Weighted Low-Rank Approximation

Abstract

We prove a convergence theorem for stochastic gradient descents on manifolds with adaptive learning rate and apply it to the weighted low-rank approximation problem.

Paper Structure

This paper contains 12 sections, 23 theorems, 83 equations, 2 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Stochastic Gradient Descents with Adaptive Learning Rates in $\mathbb{R}^n$
  3. Overview and Main Contributions
  4. A Convergent Theorem
  5. Relevant Concepts
  6. The Convergent Theorem
  7. An Application: Weighted Low-rank Approximation
  8. The Weighted Low-Rank Approximation Problem
  9. The Random Function and the Retraction
  10. The $\kappa$-confinement
  11. The Algorithm
  12. Numerical Results

Key Result

Lemma 2.6

Assume that: Let Then $\rho$ is a $\kappa$-confinement of $f$.

Figures (2)

  • Figure 1: The performance profile of the Adaptive and the Deterministic SGD.
  • Figure 2: The performance profiles of the Adaptive and the Deterministic SGD for different values of $\lambda$.

Theorems & Definitions (53)

  • Definition 2.1
  • Definition 2.2
  • Definition 2.3
  • Remark 2.4
  • Definition 2.5: $\kappa$-confinement
  • Lemma 2.6
  • proof
  • Theorem 2.7
  • Lemma 2.8
  • proof
  • ...and 43 more