Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Error bound and exact penalty method for optimization problems with nonnegative orthogonal constraint

Yitian Qian, Shaohua Pan, Lianghai Xiao

TL;DR

This paper addresses optimization under the nonnegative orthogonal constraint by deriving explicit error bounds that tie feasibility to constraint violations and establishing global exact penalty properties for penalties based on the elementwise ℓ1 distance to the nonnegative cone and its Moreau envelope. It proves Lipschitz-type error bounds (local for n>r>1, global for n>r=1 or n=r) and shows that, under mild calmness, the corresponding penalty problems share the global minimizers with the original problem, provided global optima have no zero rows. A practical, retraction-based proximal-gradient approach (SEPPG+/SEPPG0) is developed to solve a sequence of smooth penalty subproblems, with convergence to stationary points of the original problem. Extensive numerical experiments on QAPs, graph matching, projection, and ONMF demonstrate improved solution quality over state-of-the-art exact-penalty and ALM methods, albeit with modestly higher computation times. Overall, the work offers rigorous theory and a scalable algorithmic framework for nonconvex optimization on the Stiefel manifold with nonnegative constraints, with tangible impact on combinatorial-like problems in data analysis and computer vision.

Abstract

This paper is concerned with a class of optimization problems with the nonnegative orthogonal constraint, in which the objective function is $L$-smooth on an open set containing the Stiefel manifold ${\rm St}(n,r)$. We derive a locally Lipschitzian error bound for the feasible points without zero rows when $n>r>1$, and when $n>r=1$ or $n=r$ achieve a global Lipschitzian error bound. Then, we show that the penalty problem induced by the elementwise $\ell_1$-norm distance to the nonnegative cone is a global exact penalty, and so is the one induced by its Moreau envelope under a lower second-order calmness of the objective function. A practical penalty algorithm is developed by solving approximately a series of smooth penalty problems with a retraction-based nonmonotone line-search proximal gradient method, and any cluster point of the generated sequence is shown to be a stationary point of the original problem. Numerical comparisons with the ALM \citep{Wen13} and the exact penalty method \citep{JiangM22} indicate that our penalty method has an advantage in terms of the quality of solutions despite taking a little more time.

Error bound and exact penalty method for optimization problems with nonnegative orthogonal constraint

TL;DR

This paper addresses optimization under the nonnegative orthogonal constraint by deriving explicit error bounds that tie feasibility to constraint violations and establishing global exact penalty properties for penalties based on the elementwise ℓ1 distance to the nonnegative cone and its Moreau envelope. It proves Lipschitz-type error bounds (local for n>r>1, global for n>r=1 or n=r) and shows that, under mild calmness, the corresponding penalty problems share the global minimizers with the original problem, provided global optima have no zero rows. A practical, retraction-based proximal-gradient approach (SEPPG+/SEPPG0) is developed to solve a sequence of smooth penalty subproblems, with convergence to stationary points of the original problem. Extensive numerical experiments on QAPs, graph matching, projection, and ONMF demonstrate improved solution quality over state-of-the-art exact-penalty and ALM methods, albeit with modestly higher computation times. Overall, the work offers rigorous theory and a scalable algorithmic framework for nonconvex optimization on the Stiefel manifold with nonnegative constraints, with tangible impact on combinatorial-like problems in data analysis and computer vision.

Abstract

This paper is concerned with a class of optimization problems with the nonnegative orthogonal constraint, in which the objective function is -smooth on an open set containing the Stiefel manifold . We derive a locally Lipschitzian error bound for the feasible points without zero rows when , and when or achieve a global Lipschitzian error bound. Then, we show that the penalty problem induced by the elementwise -norm distance to the nonnegative cone is a global exact penalty, and so is the one induced by its Moreau envelope under a lower second-order calmness of the objective function. A practical penalty algorithm is developed by solving approximately a series of smooth penalty problems with a retraction-based nonmonotone line-search proximal gradient method, and any cluster point of the generated sequence is shown to be a stationary point of the original problem. Numerical comparisons with the ALM \citep{Wen13} and the exact penalty method \citep{JiangM22} indicate that our penalty method has an advantage in terms of the quality of solutions despite taking a little more time.

Paper Structure

This paper contains 23 sections, 21 theorems, 73 equations, 2 figures, 7 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related works
  3. Our contributions
  4. Notation
  5. Preliminaries
  6. Retractions on manifold
  7. Subdifferentials of nonsmooth functions on manifold
  8. KL property on Riemannian manifold
  9. Error bound and global exact penalties
  10. Exact penalty method for problem \ref{['orth-prob']}
  11. Convergence analysis of Algorithm \ref{['PenAlg']}
  12. PGM on manifold for solving subproblem \ref{['EP-subprob']}
  13. Numerical experiments
  14. Implementation details of SEPPG+ and SEPPG0
  15. ALM and its implementation
  16. ...and 8 more sections

Key Result

Lemma 1

Fix any $\gamma>0$. Then, for any $X\in\mathbb{R}^{n\times r}$, $\mathcal{P}_{\!\gamma}\vartheta(X)=\min(X\!+\!\gamma,\max(X,0))$ and where $J_1(X):=\{(i,j)\in[n]\times[r]\ |\ X_{ij}\in[-\gamma,0]\}$ and $J_2(X):=\{(i,j)\in[n]\times[r]\ |\ X_{ij}<-\gamma\}$.

Figures (2)

  • Figure 1: Example frame pairs for CMU house dataset generated by SEPPG+
  • Figure 2: Performance of solvers on (a) graph pairs with perfect 30 nodes (b) graph pairs with 5 nodes removed randomly

Theorems & Definitions (51)

  • Lemma 1
  • Definition 1
  • Remark 1
  • Lemma 2
  • Definition 2
  • Remark 2
  • Lemma 3
  • proof
  • Corollary 1
  • Definition 3
  • ...and 41 more