Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stochastic Optimization under Hidden Convexity

Ilyas Fatkhullin, Niao He, Yifan Hu

TL;DR

This work examines the basic projected stochastic (sub-) gradient methods for solving constrained stochastic optimization problems under hidden convexity, and provides the first sample complexity guarantees for global convergence in smooth and non-smooth settings.

Abstract

In this work, we consider constrained stochastic optimization problems under hidden convexity, i.e., those that admit a convex reformulation via non-linear (but invertible) map $c(\cdot)$. A number of non-convex problems ranging from optimal control, revenue and inventory management, to convex reinforcement learning all admit such a hidden convex structure. Unfortunately, in the majority of applications considered, the map $c(\cdot)$ is unavailable or implicit; therefore, directly solving the convex reformulation is not possible. On the other hand, the stochastic gradients with respect to the original variable are often easy to obtain. Motivated by these observations, we examine the basic projected stochastic (sub-) gradient methods for solving such problems under hidden convexity. We provide the first sample complexity guarantees for global convergence in smooth and non-smooth settings. Additionally, in the smooth setting, we improve our results to the last iterate convergence in terms of function value gap using the momentum variant of projected stochastic gradient descent.

Stochastic Optimization under Hidden Convexity

TL;DR

This work examines the basic projected stochastic (sub-) gradient methods for solving constrained stochastic optimization problems under hidden convexity, and provides the first sample complexity guarantees for global convergence in smooth and non-smooth settings.

Abstract

In this work, we consider constrained stochastic optimization problems under hidden convexity, i.e., those that admit a convex reformulation via non-linear (but invertible) map . A number of non-convex problems ranging from optimal control, revenue and inventory management, to convex reinforcement learning all admit such a hidden convex structure. Unfortunately, in the majority of applications considered, the map is unavailable or implicit; therefore, directly solving the convex reformulation is not possible. On the other hand, the stochastic gradients with respect to the original variable are often easy to obtain. Motivated by these observations, we examine the basic projected stochastic (sub-) gradient methods for solving such problems under hidden convexity. We provide the first sample complexity guarantees for global convergence in smooth and non-smooth settings. Additionally, in the smooth setting, we improve our results to the last iterate convergence in terms of function value gap using the momentum variant of projected stochastic gradient descent.
Paper Structure (26 sections, 22 theorems, 74 equations, 1 figure)

This paper contains 26 sections, 22 theorems, 74 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Related work
  3. Notations and organization
  4. Hidden Convex Problem Class
  5. Non-linear least squares nocedal1999numericalnesterov2006cubicdrusvyatskiy2019efficiency
  6. Minimizing posinomial functions boyd2007_GP_tutorialduffin1967geometric
  7. System level synthesis in optimal control anderson_2019_SLS
  8. Convex reinforcement learning zhang2020variational
  9. Revenue management and inventory control chen2022efficientchen2022network
  10. Properties of Hidden Convex Optimization
  11. Globally optimal solution
  12. Connections with gradient dominated functions
  13. Key inequalities for analysis of gradient methods
  14. Stochastic Subgradient Method
  15. Hidden Convex Setting
  16. ...and 11 more sections

Key Result

Proposition 1

Let $F(\cdot)$ be weakly convex and hidden convex on $\mathcal{X}$ with $\bar{x} \in \mathcal{X}$ being its stationary point. If the map $c(\cdot)$ is differentiable at $\bar{x}$ , then $\bar{x}$ is a global minimum, i.e., $F(\bar{x}) \leq F(x)$ for any $x \in \mathcal{X}$.

Figures (1)

  • Figure 1: The contour plots of the functions $F(x) = \frac{1}{4} (x_1 - 1)^2 + \frac{1}{2} (2 x_1^2 - x_2 - 1)^2$ (top), and $F(x) = \max\left\{\frac{1}{4} | x_1 - 1 | , \frac{1}{2} | 2 x_1^2 - x_2 - 1 | \right\}$ (bottom), $x = (x_1, x_2)^{\top}$ The left plots present the contour plots in the original space $\mathcal{X}$ and the right plots illustrate the reformulated space $\mathcal{U}$. The red star denotes the global minimum.

Theorems & Definitions (40)

  • Definition 1
  • Proposition 1
  • proof
  • Proposition 2
  • proof
  • Proposition 3
  • proof
  • Lemma 1: Lemma 3.3 in davis2019stochastic
  • Theorem 1
  • proof
  • ...and 30 more