Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A theory on the absence of spurious solutions for nonconvex and nonsmooth optimization

Cedric Josz, Yi Ouyang, Richard Y. Zhang, Javad Lavaei, Somayeh Sojoudi

TL;DR

The paper introduces the notion of global functions, a class of continuous optimization objectives whose local minima are always global, and develops a theory around their composition, change of variables, and convergence properties. It then applies this framework to nonconvex, nonsmooth tensor problems, showing that a rank-one tensor discrepancy under the $\ell_1$-norm (and related $\ell_p$-norms) is globally well-behaved in the sense of lacking spurious local minima, with a compact convergence argument establishing weak globality. This yields practical guarantees for solving rank-one tensor decomposition problems using local search and provides linear-programming reformulations that preserve globality under certain transformations. The results offer a new analytic tool for nonsmooth optimization and a theoretical justification for the use of $\ell_1$- and $\ell_\infty$-based formulations in robust nonconvex recovery tasks.

Abstract

We study the set of continuous functions that admit no spurious local optima (i.e. local minima that are not global minima) which we term \textit{global functions}. They satisfy various powerful properties for analyzing nonconvex and nonsmooth optimization problems. For instance, they satisfy a theorem akin to the fundamental uniform limit theorem in the analysis regarding continuous functions. Global functions are also endowed with useful properties regarding the composition of functions and change of variables. Using these new results, we show that a class of nonconvex and nonsmooth optimization problems arising in tensor decomposition applications are global functions. This is the first result concerning nonconvex methods for nonsmooth objective functions. Our result provides a theoretical guarantee for the widely-used $\ell_1$ norm to avoid outliers in nonconvex optimization.

A theory on the absence of spurious solutions for nonconvex and nonsmooth optimization

TL;DR

The paper introduces the notion of global functions, a class of continuous optimization objectives whose local minima are always global, and develops a theory around their composition, change of variables, and convergence properties. It then applies this framework to nonconvex, nonsmooth tensor problems, showing that a rank-one tensor discrepancy under the -norm (and related -norms) is globally well-behaved in the sense of lacking spurious local minima, with a compact convergence argument establishing weak globality. This yields practical guarantees for solving rank-one tensor decomposition problems using local search and provides linear-programming reformulations that preserve globality under certain transformations. The results offer a new analytic tool for nonsmooth optimization and a theoretical justification for the use of - and -based formulations in robust nonconvex recovery tasks.

Abstract

We study the set of continuous functions that admit no spurious local optima (i.e. local minima that are not global minima) which we term \textit{global functions}. They satisfy various powerful properties for analyzing nonconvex and nonsmooth optimization problems. For instance, they satisfy a theorem akin to the fundamental uniform limit theorem in the analysis regarding continuous functions. Global functions are also endowed with useful properties regarding the composition of functions and change of variables. Using these new results, we show that a class of nonconvex and nonsmooth optimization problems arising in tensor decomposition applications are global functions. This is the first result concerning nonconvex methods for nonsmooth objective functions. Our result provides a theoretical guarantee for the widely-used norm to avoid outliers in nonconvex optimization.

Paper Structure

This paper contains 14 sections, 12 theorems, 54 equations, 6 figures.

Table of Contents

  1. Introduction
  2. Notion of global function
  3. Application to tensor decomposition
  4. Conclusion
  5. Appendix
  6. Experiment settings
  7. Illustrations
  8. Proof of Proposition \ref{['prop:composition']}
  9. Proof of Proposition \ref{['prop:change']}
  10. Proof of Proposition \ref{['prop:definition']}
  11. Proof of Proposition \ref{['prop:weakly global']}
  12. Proof of Proposition \ref{['prop:compact']}
  13. Proof of Lemma \ref{['lemma:clarke']}
  14. Proof of Lemma \ref{['lemma:box']}

Key Result

Proposition 1.1

The function $f_{1}: \mathbb{R}^n \longrightarrow \mathbb{R}$ defined as has no spurious local minima.

Figures (6)

  • Figure 1: Experiments with sparse noise
  • Figure 2: Convergence of a sequence of global functions
  • Figure 3: All strict local minima are global minima in the sense of points but not in the sense of sets.
  • Figure 4: Examples of global functions
  • Figure 5: Compact convergence of global functions implies that strict local minima are global
  • ...and 1 more figures

Theorems & Definitions (26)

  • Proposition 1.1
  • Remark 1.1
  • Definition 2.1
  • Definition 2.2
  • Definition 2.3
  • Proposition 2.1: Composition of functions
  • Proposition 2.2: Change of variables
  • Definition 2.4
  • Proposition 2.3
  • Proposition 2.4
  • ...and 16 more