Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Near-optimal Method for Linearly Constrained Composite Non-convex Non-smooth Problems

Wei Liu, Qihang Lin, Yangyang Xu

TL;DR

This work develops PG-RPD, an inexact proximal gradient method with a recovering primal–dual subroutine for solving linearly constrained composite nonconvex nonsmooth problems. By splitting the objective and solving subproblems via a dual-based APG procedure, the algorithm achieves near-optimal oracle complexity, matching known lower bounds up to logarithmic factors without requiring bounded domains. Theoretical results provide explicit outer and inner iteration bounds, computable parameters, and conditions under which the complexity matches the lower bound; numerical experiments corroborate superior performance against ADMM and PALM in ill-conditioned settings. The approach demonstrates robustness, practical computability, and potential for broad applicability to constrained nonconvex optimization with linear constraints and nonsmooth regularizers.

Abstract

We study first-order methods (FOMs) for solving \emph{composite nonconvex nonsmooth} optimization with linear constraints. Recently, the lower complexity bounds of FOMs on finding an ($\varepsilon,\varepsilon$)-KKT point of the considered problem is established in \cite{liu2025lowercomplexityboundsfirstorder}. However, optimization algorithms that achieve this lower bound had not been developed. In this paper, we propose an inexact proximal gradient method, where subproblems are solved using a recovering primal-dual procedure. Without making the bounded domain assumption, we establish that the oracle complexity of the proposed method, for finding an ($\varepsilon,\varepsilon$)-KKT point of the considered problem, matches the lower bounds up to a logarithmic factor. Consequently, in terms of the complexity, our algorithm outperforms all existing methods. We demonstrate the advantages of our proposed algorithm over the (linearized) alternating direction method of multipliers and the (proximal) augmented Lagrangian method in the numerical experiments.

A Near-optimal Method for Linearly Constrained Composite Non-convex Non-smooth Problems

TL;DR

This work develops PG-RPD, an inexact proximal gradient method with a recovering primal–dual subroutine for solving linearly constrained composite nonconvex nonsmooth problems. By splitting the objective and solving subproblems via a dual-based APG procedure, the algorithm achieves near-optimal oracle complexity, matching known lower bounds up to logarithmic factors without requiring bounded domains. Theoretical results provide explicit outer and inner iteration bounds, computable parameters, and conditions under which the complexity matches the lower bound; numerical experiments corroborate superior performance against ADMM and PALM in ill-conditioned settings. The approach demonstrates robustness, practical computability, and potential for broad applicability to constrained nonconvex optimization with linear constraints and nonsmooth regularizers.

Abstract

We study first-order methods (FOMs) for solving \emph{composite nonconvex nonsmooth} optimization with linear constraints. Recently, the lower complexity bounds of FOMs on finding an ()-KKT point of the considered problem is established in \cite{liu2025lowercomplexityboundsfirstorder}. However, optimization algorithms that achieve this lower bound had not been developed. In this paper, we propose an inexact proximal gradient method, where subproblems are solved using a recovering primal-dual procedure. Without making the bounded domain assumption, we establish that the oracle complexity of the proposed method, for finding an ()-KKT point of the considered problem, matches the lower bounds up to a logarithmic factor. Consequently, in terms of the complexity, our algorithm outperforms all existing methods. We demonstrate the advantages of our proposed algorithm over the (linearized) alternating direction method of multipliers and the (proximal) augmented Lagrangian method in the numerical experiments.

Paper Structure

This paper contains 20 sections, 9 theorems, 96 equations, 3 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Algorithm framework
  3. Contributions
  4. Notations and definitions
  5. Organizations
  6. Related work
  7. Convergence and complexity analysis of Algorithm \ref{['alg:ipganormal']}
  8. Number of outer iterations for finding a stationary point
  9. Number of inner iterations for finding ${\mathbf{z}}^{(k+1)}$ satisfying \ref{['eq:boundz-2']}
  10. Total number of inner iterations (oracles)
  11. A computable value of $i_k$ and $j_k$ under Assumption \ref{['ass:polyhedralg']}
  12. Oracle complexity matching the lower bound
  13. Numerical Experiments
  14. Implementation details
  15. Comparisons among PG-RPD, ADMM, and PALM
  16. ...and 5 more sections

Key Result

Lemma 3.1

\newlabellem:barxyz-2 Suppose that Assumption assume:problemsetup holds. For any $\sigma>0$ and any $\overline{{\mathbf{z}}}^{(k+1)}\in\Omega^{(k+1)}$, let $(\overline{{\mathbf{x}}}^{(k+1)}, \overline{{\mathbf{y}}}^{(k+1)})$ be the optimal solution of the strongly convex problem where ${\mathcal{L}}_k$ and $\Omega^{(k+1)}$ are defined in eq:lag-func and eq:sublb-2, respectively. Then it holds tha

Figures (3)

  • Figure 4.1: Comparisons among PG-RPD, ADMM in melo2017iteration3, and PALM in rockafellar1976augmented on solving instances of \ref{['eq:test']} with weak convexity modulus $\rho=0.1$
  • Figure 4.2: Comparisons among PG-RPD, ADMM in melo2017iteration3, and PALM in rockafellar1976augmented on solving instances of \ref{['eq:test']} with weak convexity modulus $\rho=1$
  • Figure 4.3: Comparisons among PG-RPD, ADMM in melo2017iteration3, and PALM in rockafellar1976augmented on solving instances of \ref{['eq:test']} with weak convexity modulus $\rho=5$

Theorems & Definitions (26)

  • Definition 1.1
  • Lemma 3.1
  • proof
  • Remark 3.1
  • Lemma 3.2
  • proof
  • Theorem 3.3
  • proof
  • Remark 3.2: Boundedness of the sequence
  • Theorem 3.4
  • ...and 16 more