Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Nonsmooth Convex Optimization using the Specular Gradient Method with Root-Linear Convergence

Kiyuob Jung, Jehan Oh

TL;DR

This paper addresses 1D nonsmooth convex minimization using the specular gradient method (SGM), a subgradient-method variant built from the specular derivative $f^{\wedge}$ that achieves $R$-linear convergence under convexity alone. It introduces an implicit variant (ISGM) that avoids explicit computation of $f^{\wedge}$ by leveraging $f'_+ + f'_-$ signs, and develops a corresponding convergence theory based on Shor’s framework. The paper also builds convex analysis with specular derivatives, showing $f^{\wedge}$ is a subderivative and that convexity corresponds to monotone specular derivatives, enabling practical algorithms. Numerical experiments illustrate that ISGM can outperform standard subgradient methods in 1D while preserving agreement with SM when smoothness is present, and the authors discuss extensions to higher dimensions.

Abstract

In this paper, we find the special case of the subgradient method minimizing a one-dimensional real-valued function, which we term the specular gradient method, that converges root-linearly without any additional assumptions except the convexity. Furthermore, we suggest a way to implement the specular gradient method without explicitly calculating specular derivatives.

Nonsmooth Convex Optimization using the Specular Gradient Method with Root-Linear Convergence

TL;DR

This paper addresses 1D nonsmooth convex minimization using the specular gradient method (SGM), a subgradient-method variant built from the specular derivative that achieves -linear convergence under convexity alone. It introduces an implicit variant (ISGM) that avoids explicit computation of by leveraging signs, and develops a corresponding convergence theory based on Shor’s framework. The paper also builds convex analysis with specular derivatives, showing is a subderivative and that convexity corresponds to monotone specular derivatives, enabling practical algorithms. Numerical experiments illustrate that ISGM can outperform standard subgradient methods in 1D while preserving agreement with SM when smoothness is present, and the authors discuss extensions to higher dimensions.

Abstract

In this paper, we find the special case of the subgradient method minimizing a one-dimensional real-valued function, which we term the specular gradient method, that converges root-linearly without any additional assumptions except the convexity. Furthermore, we suggest a way to implement the specular gradient method without explicitly calculating specular derivatives.
Paper Structure (8 sections, 12 theorems, 70 equations, 5 figures, 4 tables, 2 algorithms)

This paper contains 8 sections, 12 theorems, 70 equations, 5 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Convex Analysis with Specular Derivatives
  4. Specular Gradient Method
  5. Algorithm
  6. Convergence Analysis
  7. Revised Algorithm
  8. Numerical Examples

Key Result

Lemma 2.1

Let $\alpha, \beta \in \mathbb{R}$ be such that $\beta \leq \alpha$ and $\alpha + \beta \neq 0$. Then we have

Figures (5)

  • Figure 1: The first-order convexity condition in the specular derivative sense
  • Figure 2: Comparison of the subgradient method and the specular gradient method for the objective function $f$
  • Figure 3: Comparison of the subgradient method and the specular gradient method for the objective function $g$
  • Figure 4: Comparison of the subgradient method and the specular gradient method for the objective function $H_{\delta}$
  • Figure 5: Comparison of the subgradient method and the specular gradient method for the objective function $j$

Theorems & Definitions (24)

  • Lemma 2.1
  • proof
  • Proposition 2.2
  • proof
  • Proposition 2.3
  • proof
  • Proposition 2.4
  • Corollary 2.5
  • proof
  • Theorem 3.1
  • ...and 14 more