Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Steepest Gradient Method with Nonmonotone Adaptive Step-sizes for the Nonconvex Minimax and Multi-Objective Optimization Problems

Nguyen Duc Anh, Tran Ngoc Thang

TL;DR

This work tackles nonconvex finite minimax problems and nonconvex multiobjective optimization by introducing a steepest-gradient method with nonmonotone adaptive step-sizes that avoids line searches. The algorithm is analyzed under nonconvex, quasiconvex, and pseudoconvex component assumptions, proving convergence to stationary or Pareto-stationary points, and is extended to multiobjective problems via Tchebycheff scalarization in a reference-based framework. Convergence results mirror the single-objective case, yielding weakly efficient or Pareto stationary solutions depending on convexity assumptions, with numerical experiments validating effectiveness on convex, nonconvex, and high-dimensional problems. The approach offers computational efficiency and flexible Pareto-front coverage, making it attractive for learning and decision-making contexts where line searches are costly or impractical.

Abstract

This paper proposes a new steepest gradient descent method for solving nonconvex finite minimax problems using non-monotone adaptive step sizes and providing proof of convergence results in cases of the nonconvex, quasiconvex, and pseudoconvex differentiate component functions. The proposed method is applied using a referenced-based approach to solve the nonconvex multiobjective programming problems. The convergence to weakly efficient or Pareto stationary solutions is proved for pseudoconvex or quasiconvex multiobjective optimization problems, respectively. A variety of numerical experiments are provided for each scenario to verify the correctness of the theoretical results corresponding to the algorithms proposed for the minimax and multiobjective optimization problems.

A Steepest Gradient Method with Nonmonotone Adaptive Step-sizes for the Nonconvex Minimax and Multi-Objective Optimization Problems

TL;DR

This work tackles nonconvex finite minimax problems and nonconvex multiobjective optimization by introducing a steepest-gradient method with nonmonotone adaptive step-sizes that avoids line searches. The algorithm is analyzed under nonconvex, quasiconvex, and pseudoconvex component assumptions, proving convergence to stationary or Pareto-stationary points, and is extended to multiobjective problems via Tchebycheff scalarization in a reference-based framework. Convergence results mirror the single-objective case, yielding weakly efficient or Pareto stationary solutions depending on convexity assumptions, with numerical experiments validating effectiveness on convex, nonconvex, and high-dimensional problems. The approach offers computational efficiency and flexible Pareto-front coverage, making it attractive for learning and decision-making contexts where line searches are costly or impractical.

Abstract

This paper proposes a new steepest gradient descent method for solving nonconvex finite minimax problems using non-monotone adaptive step sizes and providing proof of convergence results in cases of the nonconvex, quasiconvex, and pseudoconvex differentiate component functions. The proposed method is applied using a referenced-based approach to solve the nonconvex multiobjective programming problems. The convergence to weakly efficient or Pareto stationary solutions is proved for pseudoconvex or quasiconvex multiobjective optimization problems, respectively. A variety of numerical experiments are provided for each scenario to verify the correctness of the theoretical results corresponding to the algorithms proposed for the minimax and multiobjective optimization problems.

Paper Structure

This paper contains 16 sections, 14 theorems, 19 equations, 3 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Notations
  4. Basic Results
  5. Steepest Gradient Algorithms with Nonmonotone Adaptive Step-sizes for Problem (MP)
  6. Minimax problem
  7. The nonmonotone adaptive steepest gradient algorithm
  8. Convergence of the algorithms
  9. Multiobjective Optimization Problem
  10. Notations
  11. Linear Scalarization
  12. Tchebycheff Scalarization
  13. Non-monotone adaptive algorithm
  14. Convergence of the algorithms
  15. Numerical Results
  16. ...and 1 more sections

Key Result

Proposition 2.1

agrawal2020disciplined Let $g_i : \mathbb{R}^n \to \mathbb{R}$, $i = \overline{1, m}$ be quasiconvex functions. Then, $G(\theta) = \max_i\{g_i(\theta), i = \overline{1, m}\}$ is quasiconvex.

Figures (3)

  • Figure 1: Results from Algorithm \ref{['algo2']} applied to Example \ref{['4.1']}
  • Figure 2: Results from Algorithm \ref{['algo2']} applied to Example \ref{['4.2']}
  • Figure 3: Results from Algorithm \ref{['algo2']} applied to Example \ref{['4.4']}

Theorems & Definitions (21)

  • Proposition 2.1
  • Lemma 2.2
  • Proposition 2.3
  • Proposition 2.4
  • Proposition 2.5
  • Theorem 2.6
  • Lemma 3.2
  • Lemma 3.3
  • Lemma 3.4
  • Theorem 3.5
  • ...and 11 more