Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Solving nonconvex optimization problems via a second order dynamical system with unbounded damping

Szilárd Csaba László

TL;DR

The paper studies a second-order dynamical system with unbounded damping (specifically, terms like λ/t^2 and γ/t) for solving the minimization of a smooth nonconvex function g. By introducing a KL-regularized Lyapunov function H(u,v) = g(u) + (1/2)‖u−v‖^2 and leveraging KL properties, the authors prove convergence of trajectories to critical points and derive convergence rates that depend on the KL exponent θ. For strongly convex g, the results yield superlinear rates e.g., ‖x(t) − x*‖ = O(e^(−a t^2)). The work also provides existence/uniqueness of strong global solutions, establishes integrability properties of higher-order derivatives, and situates the approach within a broader KL framework including semi-algebraic functions, implying broad applicability to nonconvex optimization with structured regularizers.

Abstract

In this paper we study a second order dynamical system with variable coefficients in connection to the minimization problem of a smooth nonconvex function. The convergence of the trajectories generated by the dynamical system to a critical point of the objective function is assured, provided a regularization of the objective function satisfies the Kurdyka-Łojasiewicz property. We also provide convergence rates for the trajectories generated by the dynamical system, formulated in terms of the Łojasiewicz exponent, and we show that the unbounded damping considered in our dynamical system significantly improves the convergence rates known so far in the literature, that is, instead of linear rates we obtain superlinear rates.

Solving nonconvex optimization problems via a second order dynamical system with unbounded damping

TL;DR

The paper studies a second-order dynamical system with unbounded damping (specifically, terms like λ/t^2 and γ/t) for solving the minimization of a smooth nonconvex function g. By introducing a KL-regularized Lyapunov function H(u,v) = g(u) + (1/2)‖u−v‖^2 and leveraging KL properties, the authors prove convergence of trajectories to critical points and derive convergence rates that depend on the KL exponent θ. For strongly convex g, the results yield superlinear rates e.g., ‖x(t) − x*‖ = O(e^(−a t^2)). The work also provides existence/uniqueness of strong global solutions, establishes integrability properties of higher-order derivatives, and situates the approach within a broader KL framework including semi-algebraic functions, implying broad applicability to nonconvex optimization with structured regularizers.

Abstract

In this paper we study a second order dynamical system with variable coefficients in connection to the minimization problem of a smooth nonconvex function. The convergence of the trajectories generated by the dynamical system to a critical point of the objective function is assured, provided a regularization of the objective function satisfies the Kurdyka-Łojasiewicz property. We also provide convergence rates for the trajectories generated by the dynamical system, formulated in terms of the Łojasiewicz exponent, and we show that the unbounded damping considered in our dynamical system significantly improves the convergence rates known so far in the literature, that is, instead of linear rates we obtain superlinear rates.
Paper Structure (5 sections, 12 theorems, 143 equations)

This paper contains 5 sections, 12 theorems, 143 equations.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Existence and uniqueness
  4. Convergence of trajectories
  5. Convergence rates

Key Result

Lemma 2.3

Suppose that $F:[0,+\infty)\rightarrow\mathbb{R}$ is locally absolutely continuous and bounded below and that there exists $G\in L^1([0,+\infty))$ such that for almost every $t \in [0,+\infty)$ Then there exists $\lim_{t\rightarrow \infty} F(t)\in\mathbb{R}$.

Theorems & Definitions (28)

  • Definition 2.1
  • Remark 2.2
  • Lemma 2.3
  • Lemma 2.4
  • Definition 2.5
  • Lemma 2.6
  • Definition 2.7
  • Definition 3.1
  • Theorem 3.2
  • proof
  • ...and 18 more