Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Regularized higher-order Taylor approximation methods for nonlinear least-squares

Yassine Nabou, Ion Necoara

TL;DR

This document provides a comprehensive guide to the SIAM LaTeX style and its associated templates, tools, and bibliography support. It covers class options, front matter, cross-referencing, math typesetting, and theorem-like environments, as well as how to manage supplementary materials and modern bibliographic fields. It introduces templates (ex_article, ex_shared, ex_supplement) and explains changes to improve hyperlinking, PDF bookmarks, and preprint/bibliography handling. The practical impact is a standardized, robust workflow for preparing SIAM-compliant manuscripts with consistent formatting, advanced referencing, and streamlined integration of supplementary content.

Abstract

In this paper, we develop a regularized higher-order Taylor based method for solving composite (e.g., nonlinear least-squares) problems. At each iteration, we replace each smooth component of the objective function by a higher-order Taylor approximation with an appropriate regularization, leading to a regularized higher-order Taylor approximation (RHOTA) algorithm. We derive global convergence guarantees for RHOTA algorithm. In particular, we prove stationary point convergence guarantees for the iterates generated by RHOTA, and leveraging a Kurdyka-Łojasiewicz (KL) type property of the objective function, we derive improved rates depending on the KL parameter. When the Taylor approximation is of order $2$, we present an efficient implementation of RHOTA algorithm, demonstrating that the resulting nonconvex subproblem can be effectively solved utilizing standard convex programming tools. Furthermore, we extend the scope of our investigation to include the behavior and efficacy of RHOTA algorithm in handling systems of nonlinear equations and optimization problems with nonlinear equality constraints deriving new rates under improved constraint qualifications conditions. Finally, we consider solving the phase retrieval problem with a higher-order proximal point algorithm, showcasing its rapid convergence rate for this particular application. Numerical simulations on phase retrieval and output feedback control problems also demonstrate the efficacy and performance of the proposed methods when compared to some state-of-the-art optimization methods and software.

Regularized higher-order Taylor approximation methods for nonlinear least-squares

TL;DR

This document provides a comprehensive guide to the SIAM LaTeX style and its associated templates, tools, and bibliography support. It covers class options, front matter, cross-referencing, math typesetting, and theorem-like environments, as well as how to manage supplementary materials and modern bibliographic fields. It introduces templates (ex_article, ex_shared, ex_supplement) and explains changes to improve hyperlinking, PDF bookmarks, and preprint/bibliography handling. The practical impact is a standardized, robust workflow for preparing SIAM-compliant manuscripts with consistent formatting, advanced referencing, and streamlined integration of supplementary content.

Abstract

In this paper, we develop a regularized higher-order Taylor based method for solving composite (e.g., nonlinear least-squares) problems. At each iteration, we replace each smooth component of the objective function by a higher-order Taylor approximation with an appropriate regularization, leading to a regularized higher-order Taylor approximation (RHOTA) algorithm. We derive global convergence guarantees for RHOTA algorithm. In particular, we prove stationary point convergence guarantees for the iterates generated by RHOTA, and leveraging a Kurdyka-Łojasiewicz (KL) type property of the objective function, we derive improved rates depending on the KL parameter. When the Taylor approximation is of order , we present an efficient implementation of RHOTA algorithm, demonstrating that the resulting nonconvex subproblem can be effectively solved utilizing standard convex programming tools. Furthermore, we extend the scope of our investigation to include the behavior and efficacy of RHOTA algorithm in handling systems of nonlinear equations and optimization problems with nonlinear equality constraints deriving new rates under improved constraint qualifications conditions. Finally, we consider solving the phase retrieval problem with a higher-order proximal point algorithm, showcasing its rapid convergence rate for this particular application. Numerical simulations on phase retrieval and output feedback control problems also demonstrate the efficacy and performance of the proposed methods when compared to some state-of-the-art optimization methods and software.

Paper Structure

This paper contains 29 sections, 2 theorems, 7 equations, 2 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Class options
  3. Front matter
  4. Cross references and hyperlinks
  5. Cleveref
  6. Hyperef
  7. Math and equations
  8. Theorem-like environments
  9. Tables
  10. Figures
  11. Algorithms
  12. Sections
  13. Supplemental material
  14. Template
  15. Bibliography
  16. ...and 14 more sections

Key Result

Theorem 6.1

\newlabelthm:mvt0 Suppose $f$ is a function that is continuous on the closed interval $[a,b]$. and differentiable on the open interval $(a,b)$. Then there exists a number $c$ such that $a < c < b$ and In other words, $f(b)-f(a) = f'(c)(b-a)$.

Figures (2)

  • Figure 1: Example figure using external image files.
  • Figure 2: Example PGFPLOTS figure.

Theorems & Definitions (5)

  • Theorem 6.1: Mean Value Theorem
  • Corollary 6.2
  • Proof 1
  • Claim 6.3
  • Proof 2: Proof of main theorem