Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Inexact Variable Metric Proximal Gradient-subgradient Algorithm for a Class of Fractional Optimization Problems

Lei Yang, Xiangrui Kong, Min Zhang, Yaohua Hu

TL;DR

This work tackles fractional optimization $\min_{\bm{x}\in\Omega} \frac{f(\bm{x}) + h(\bm{x})}{g(\bm{x})}$ where $f$ is convex, $h$ has a Lipschitz gradient, and $g$ is convex and positive. It introduces an inexact variable metric proximal gradient-subgradient algorithm (iVPGSA) that allows approximate subproblem solutions under a verifiable error criterion and employs a variable metric $\{H_k\}$ to adapt to problem geometry. A KL-based analysis using an auxiliary function $\Phi_{\tau}$ yields global sequential convergence and detailed convergence-rate results dependent on the KL exponent, without requiring a strict descent property. Numerical experiments on $\ell_1/\ell_2$ Lasso and constrained $\ell_1/\ell_2$ sparse optimization validate the method, showcasing superior performance to PGSA_BE and MBA and illustrating the effectiveness of a dual semi-smooth Newton solver for the subproblems.

Abstract

In this paper, we study a class of fractional optimization problems, in which the numerator of the objective is the sum of a convex function and a differentiable function with a Lipschitz continuous gradient, while the denominator is a nonsmooth convex function. This model has broad applicability and encompasses several important optimization problems in the literature. To address these problems, we propose an inexact variable metric proximal gradient-subgradient algorithm (iVPGSA), which, to our knowledge, is the first inexact proximal algorithm specifically designed for solving such type of fractional problems. By incorporating a variable metric proximal term and allowing for inexact solutions to the subproblem under a flexible error criterion, the proposed algorithm is highly adaptable to a broader range of problems while achieving favorable computational efficiency. Under mild assumptions, we establish that any accumulation point of the sequence generated by the iVPGSA is a critical point of the target problem. Moreover, we develop an improved Kurdyka-Łojasiewicz (KL)-based analysis framework to prove the global convergence of the entire sequence and characterize its convergence rate, \textit{without} requiring a strict sufficient descent property. Our results offer detailed insights into how the KL exponent and inexactness influence the convergence rate. The proposed analysis framework also has the potential to serve as a theoretical tool for studying the convergence rates of a wide range of inexact algorithms beyond the iVPGSA. Finally, some numerical experiments on the $\ell_1/\ell_2$ Lasso problem and the constrained $\ell_1/\ell_2$ sparse optimization problem are conducted to show the superior performance of the iVPGSA in comparison to existing algorithms.

An Inexact Variable Metric Proximal Gradient-subgradient Algorithm for a Class of Fractional Optimization Problems

TL;DR

This work tackles fractional optimization where is convex, has a Lipschitz gradient, and is convex and positive. It introduces an inexact variable metric proximal gradient-subgradient algorithm (iVPGSA) that allows approximate subproblem solutions under a verifiable error criterion and employs a variable metric to adapt to problem geometry. A KL-based analysis using an auxiliary function yields global sequential convergence and detailed convergence-rate results dependent on the KL exponent, without requiring a strict descent property. Numerical experiments on Lasso and constrained sparse optimization validate the method, showcasing superior performance to PGSA_BE and MBA and illustrating the effectiveness of a dual semi-smooth Newton solver for the subproblems.

Abstract

In this paper, we study a class of fractional optimization problems, in which the numerator of the objective is the sum of a convex function and a differentiable function with a Lipschitz continuous gradient, while the denominator is a nonsmooth convex function. This model has broad applicability and encompasses several important optimization problems in the literature. To address these problems, we propose an inexact variable metric proximal gradient-subgradient algorithm (iVPGSA), which, to our knowledge, is the first inexact proximal algorithm specifically designed for solving such type of fractional problems. By incorporating a variable metric proximal term and allowing for inexact solutions to the subproblem under a flexible error criterion, the proposed algorithm is highly adaptable to a broader range of problems while achieving favorable computational efficiency. Under mild assumptions, we establish that any accumulation point of the sequence generated by the iVPGSA is a critical point of the target problem. Moreover, we develop an improved Kurdyka-Łojasiewicz (KL)-based analysis framework to prove the global convergence of the entire sequence and characterize its convergence rate, \textit{without} requiring a strict sufficient descent property. Our results offer detailed insights into how the KL exponent and inexactness influence the convergence rate. The proposed analysis framework also has the potential to serve as a theoretical tool for studying the convergence rates of a wide range of inexact algorithms beyond the iVPGSA. Finally, some numerical experiments on the Lasso problem and the constrained sparse optimization problem are conducted to show the superior performance of the iVPGSA in comparison to existing algorithms.

Paper Structure

This paper contains 20 sections, 19 theorems, 138 equations, 2 figures, 2 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Notation and preliminaries
  3. An inexact variable metric proximal gradient-subgradient algorithm
  4. The KL-based convergence analysis
  5. Global sequential convergence
  6. Convergence rate analysis
  7. Further discussions on the KL exponent of $\Phi_{\tau}$
  8. Numerical experiments
  9. The $\ell_1/\ell_2$ Lasso problem
  10. A dual semi-smooth Newton method for solving the subproblem \ref{['subprob_l12lasso']}
  11. Comparison results
  12. The constrained $\ell_1/\ell_2$ sparse optimization problem
  13. A dual semi-smooth Newton method for solving the subproblem \ref{['subprob_l12cso']}
  14. Comparison results
  15. Conclusions
  16. ...and 5 more sections

Key Result

Proposition 2.1

Suppose that $h: \mathbb{R}^n \rightarrow \mathbb{R} \cup \{+\infty\}$ is a proper closed function and $\Gamma$ is a compact set. If $h \equiv \zeta$ on $\Gamma$ for some constant $\zeta$ and satisfies the KL property at each point of $\Gamma$, then there exist $\mu,\nu>0$ and $\varphi \in \Xi_{\nu} for all $\bm{x} \in \{\bm{x}\in\mathbb{R}^{n}: \mathrm{dist}(\bm{x},\,\Gamma)<\mu\} \cap \{\bm{x}\i

Figures (2)

  • Figure 1: Numerical results of PGSA_BE and iVPGSA for solving the $\ell_1/\ell_2$ Lasso problem on mpg7 and gisette from the UCI data repository.
  • Figure 2: Numerical results of MBA and iVPGSA for solving the constrained $\ell_1/\ell_2$ sparse optimization problem on mpg7 and gisette from the UCI data repository.

Theorems & Definitions (37)

  • Definition 2.1: KL property and KL function
  • Definition 2.2: KL exponent
  • Proposition 2.1: Uniformized KL property
  • Lemma 2.1: lp2017calculus
  • Lemma 2.2: p1987introduction
  • Lemma 2.3
  • proof
  • Definition 2.3: Critical point
  • Lemma 3.1
  • proof
  • ...and 27 more