Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Moving higher-order Taylor approximations method for smooth constrained minimization problems

Yassine Nabou, Ion Necoara

TL;DR

The paper develops Moving Taylor Approximation (MTA), a higher-order method for composite smooth minimization with smooth functional constraints, by replacing the objective and constraints with higher-order Taylor models plus regularization. It proves global convergence to KKT points in the nonconvex case and provides convergence rates under the KL property, with sublinear to linear behavior depending on the KL exponent; in the convex and uniformly convex cases, it yields sublinear and (potentially) linear or superlinear rates in function values. The authors also show that subproblems are implementable (especially for p,q ≤ 2) via convex optimization techniques and provide an adaptive variant that eliminates the need for Lipschitz constants. Numerical experiments illustrate that higher-order MTA variants outperform first-order SCP/MBA approaches, highlighting improved efficiency and faster residual reduction. Overall, MTA offers a scalable, higher-order framework for smooth constrained optimization with strong theoretical guarantees and practical solvability.

Abstract

In this paper we develop a higher-order method for solving composite (non)convex minimization problems with smooth (non)convex functional constraints. At each iteration our method approximates the smooth part of the objective function and of the constraints by higher-order Taylor approximations, leading to a moving Taylor approximation method (MTA). We present convergence guarantees for MTA algorithm for both, nonconvex and convex problems. In particular, when the objective and the constraints are nonconvex functions, we prove that the sequence generated by MTA algorithm converges globally to a KKT point. Moreover, we derive convergence rates in the iterates when the problem data satisfy the Kurdyka-Lojasiewicz (KL) property. Further, when the objective function is (uniformly) convex and the constraints are also convex, we provide (linear/superlinear) sublinear convergence rates for our algorithm. Finally, we present an efficient implementation of the proposed algorithm and compare it with existing methods from the literature.

Moving higher-order Taylor approximations method for smooth constrained minimization problems

TL;DR

The paper develops Moving Taylor Approximation (MTA), a higher-order method for composite smooth minimization with smooth functional constraints, by replacing the objective and constraints with higher-order Taylor models plus regularization. It proves global convergence to KKT points in the nonconvex case and provides convergence rates under the KL property, with sublinear to linear behavior depending on the KL exponent; in the convex and uniformly convex cases, it yields sublinear and (potentially) linear or superlinear rates in function values. The authors also show that subproblems are implementable (especially for p,q ≤ 2) via convex optimization techniques and provide an adaptive variant that eliminates the need for Lipschitz constants. Numerical experiments illustrate that higher-order MTA variants outperform first-order SCP/MBA approaches, highlighting improved efficiency and faster residual reduction. Overall, MTA offers a scalable, higher-order framework for smooth constrained optimization with strong theoretical guarantees and practical solvability.

Abstract

In this paper we develop a higher-order method for solving composite (non)convex minimization problems with smooth (non)convex functional constraints. At each iteration our method approximates the smooth part of the objective function and of the constraints by higher-order Taylor approximations, leading to a moving Taylor approximation method (MTA). We present convergence guarantees for MTA algorithm for both, nonconvex and convex problems. In particular, when the objective and the constraints are nonconvex functions, we prove that the sequence generated by MTA algorithm converges globally to a KKT point. Moreover, we derive convergence rates in the iterates when the problem data satisfy the Kurdyka-Lojasiewicz (KL) property. Further, when the objective function is (uniformly) convex and the constraints are also convex, we provide (linear/superlinear) sublinear convergence rates for our algorithm. Finally, we present an efficient implementation of the proposed algorithm and compare it with existing methods from the literature.
Paper Structure (12 sections, 12 theorems, 113 equations, 1 figure, 2 tables, 2 algorithms)

This paper contains 12 sections, 12 theorems, 113 equations, 1 figure, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Notations and preliminaries
  3. A moving Taylor approximation method
  4. Nonconvex convergence analysis
  5. Convergence rate to KKT points
  6. Adaptive MTA
  7. Better convergence under KL
  8. Convex convergence analysis
  9. Uniform convex convergence analysis
  10. Efficient solution of subproblem for (non)convex problems
  11. Numerical illustrations
  12. Conclusions

Key Result

Lemma 4.1

\newlabelth:10 Let Assumptions ass:1, ass:2, ass:3, and th:kkt hold and the sequence $(x_k)_{k\geq 0}$ be generated by MTA algorithm with $x_0 \in \@fontswitch{}{\mathcal{}} F$, $M_p>L_p, M>0$ and $M_q^i \geq L_q^i + \textcolor{black}{\eta_3}$ for all $i=1:m$. Then, we have:

Figures (1)

  • Figure 1: Behaviour of residual function for MTA with $q=2,p=1$ and with $q=p=2$, MBA and SCP along iterations (left) and time in sec (right): $n=100, m=10$.

Theorems & Definitions (30)

  • Definition 2.1
  • Lemma 4.1
  • Proof 1
  • Lemma 4.2
  • Proof 2
  • Theorem 4.3
  • Proof 3
  • Remark 4.4
  • Lemma 4.5
  • Proof 4
  • ...and 20 more