Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Variational Analysis of a Nonconvex and Nonsmooth Optimization Problem: An Introduction

Johannes O. Royset

TL;DR

This survey presents variational analysis tools for handling nonconvex and nonsmooth optimization in a finite-dimensional, composite setting $\min_{x\in X} h(G(x))$, where $X$ is closed and convex, $G$ is $C^1$, and $h$ is convex. It develops first-order optimality via subgradients and chain rules, then addresses approximations through epi- and graphical convergence to justify consistent approximations in practice. The article surveys algorithmic frameworks, including epigraphical and dual reformulations, and introduces proximal composite methods, with a detailed discussion of Rockafellian relaxations, Lagrangian duality, and augmented formulations. A substantial portion is devoted to second-order theory—coderivatives and tilt-stability—to analyze stability and guide advanced algorithms. The work highlights how these variational tools unify constrained and unconstrained problems, accommodate data ambiguity, and support robust algorithm design for nonsmooth, nonconvex optimization.

Abstract

Variational analysis provides the theoretical foundations and practical tools for constructing optimization algorithms without being restricted to smooth or convex problems. We survey the central concepts in the context of a concrete but broadly applicable problem class from composite optimization in finite dimensions. While prioritizing accessibility over mathematical details, we introduce subgradients of arbitrary functions and the resulting optimality conditions, describe approximations and the need for going beyond pointwise and uniform convergence, and summarize proximal methods. We derive dual problems from parametrization of the actual problem and the resulting relaxations. The paper ends with an introduction to second-order theory and its role in stability analysis of optimization problems.

Variational Analysis of a Nonconvex and Nonsmooth Optimization Problem: An Introduction

TL;DR

This survey presents variational analysis tools for handling nonconvex and nonsmooth optimization in a finite-dimensional, composite setting , where is closed and convex, is , and is convex. It develops first-order optimality via subgradients and chain rules, then addresses approximations through epi- and graphical convergence to justify consistent approximations in practice. The article surveys algorithmic frameworks, including epigraphical and dual reformulations, and introduces proximal composite methods, with a detailed discussion of Rockafellian relaxations, Lagrangian duality, and augmented formulations. A substantial portion is devoted to second-order theory—coderivatives and tilt-stability—to analyze stability and guide advanced algorithms. The work highlights how these variational tools unify constrained and unconstrained problems, accommodate data ambiguity, and support robust algorithm design for nonsmooth, nonconvex optimization.

Abstract

Variational analysis provides the theoretical foundations and practical tools for constructing optimization algorithms without being restricted to smooth or convex problems. We survey the central concepts in the context of a concrete but broadly applicable problem class from composite optimization in finite dimensions. While prioritizing accessibility over mathematical details, we introduce subgradients of arbitrary functions and the resulting optimality conditions, describe approximations and the need for going beyond pointwise and uniform convergence, and summarize proximal methods. We derive dual problems from parametrization of the actual problem and the resulting relaxations. The paper ends with an introduction to second-order theory and its role in stability analysis of optimization problems.

Paper Structure

This paper contains 8 sections, 13 theorems, 83 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Examples
  3. Variational Analysis: Basic Concepts
  4. First-Order Optimality Conditions
  5. Approximations and Consistency
  6. Algorithms
  7. Problem Relaxation and Duality
  8. Second-Order Theory

Key Result

Proposition 3.1

(normal cone to polyhedral sets). Suppose that where $A$ and $D$ are $m\times n$ and $q\times n$ matrices, respectively. For any $\bar{x}\in C$, where $\mathbb{A}(\bar{x}) = \{i~|~\langle D_i, \bar{x}\rangle = d_i\}$, $D_i$ is the $i$th row of $D$, and $d_i$ is the $i$th component of $d$.

Figures (4)

  • Figure 1: Regular normal vectors and normal vectors (solid arrows) at a point of smoothness (left), at a kink (middle), and at a cusp (right).
  • Figure 2: Normal vectors of $\mathop{\mathrm{epi}}\nolimits f$ defining the two subgradients $\partial f(x') = \{-4, -1/2\}$ and the interval of subgradients $\partial f(\bar{x}) = [1/2, 2]$.
  • Figure 3: The graph of a set-valued mapping $S:\mathbb{R}^n\;{\hbox{$\rightarrow$}} \hbox{$\rightarrow$}\;\mathbb{R}^m$, its domain $\mathop{\mathrm{dom}}\nolimits S$, and the solution set $S^{-1}(\bar{y})$ to the generalized equation $\bar{y} \in S(x)$.
  • Figure 4: Normal cones of $\mathop{\mathrm{gph}}\nolimits S$ at the points $(0,-1)$, $(0,0)$, and $(0,2)$ define coderivatives.

Theorems & Definitions (20)

  • Example 2.1
  • Example 2.2
  • Example 2.3
  • Example 2.4
  • Example 2.5
  • Proposition 3.1
  • Theorem 4.1
  • Theorem 4.2
  • Theorem 4.3
  • Theorem 5.1
  • ...and 10 more