Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SPICE: Scaling-Aware Prediction Correction Methods with a Free Convergence Rate for Nonlinear Convex Optimization

Sai Wang

Abstract

Recently, the prediction-correction method has been developed to solve nonlinear convex optimization problems. However, its convergence rate is often poor since large regularization parameters are set to ensure convergence conditions. In this paper, the scaling-aware prediction correction (\textsf{Spice}) method is proposed to achieve a free convergence rate. This method adopts a novel scaling technique that adjusts the weight of the objective and constraint functions. The theoretical analysis demonstrates that increasing the scaling factor for the objective function or decreasing the scaling factor for constraint functions significantly enhances the convergence rate of the prediction correction method. In addition, the \textsf{Spice} method is further extended to solve separable variable nonlinear convex optimization. By employing different scaling factors as functions of the iterations, the \textsf{Spice} method achieves convergence rates of $\mathcal{O}(1/(t+1))$, $\mathcal{O}(1/[e^{t}(t+1)])$, and $\mathcal{O}(1/(t+1)^{t+1})$. Numerical experiments further validate the theoretical findings, demonstrating the effectiveness of the \textsf{Spice} method in practice.

SPICE: Scaling-Aware Prediction Correction Methods with a Free Convergence Rate for Nonlinear Convex Optimization

Abstract

Recently, the prediction-correction method has been developed to solve nonlinear convex optimization problems. However, its convergence rate is often poor since large regularization parameters are set to ensure convergence conditions. In this paper, the scaling-aware prediction correction (\textsf{Spice}) method is proposed to achieve a free convergence rate. This method adopts a novel scaling technique that adjusts the weight of the objective and constraint functions. The theoretical analysis demonstrates that increasing the scaling factor for the objective function or decreasing the scaling factor for constraint functions significantly enhances the convergence rate of the prediction correction method. In addition, the \textsf{Spice} method is further extended to solve separable variable nonlinear convex optimization. By employing different scaling factors as functions of the iterations, the \textsf{Spice} method achieves convergence rates of , , and . Numerical experiments further validate the theoretical findings, demonstrating the effectiveness of the \textsf{Spice} method in practice.

Paper Structure

This paper contains 25 sections, 17 theorems, 110 equations, 2 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Variational Inequality
  4. Scaling Technique
  5. Scaling-Aware Prediction Correction Method
  6. PPA-Like Prediction Scheme
  7. Matrix-Driven Correction Scheme
  8. Sequence Convergence Analysis
  9. Ergodic Convergence Analysis
  10. Solving Convex Problems with Equality Constraints
  11. Nonlinear Convex Problems with Separable Variables
  12. PPA-Like Prediction Scheme
  13. Matrix-Driven Correction Scheme
  14. Convergence Analysis
  15. Solving Convex Problems with Equality Constraints
  16. ...and 10 more sections

Key Result

Lemma 2.1

Let $\mathcal{X}\subset \mathbb{R}^{n}$ be a closed convex set, with $f(\mathbf{x})$ and $h(\mathbf{x})$ as convex functions, where $h(\mathbf{x})$ is differentiable. Assume the minimization problem $\min\{ f(\mathbf{x})+ h(\mathbf{x})\mid \mathbf{x}\in \mathcal{X}\}$ has a nonempty solution set. Th if and only if

Figures (2)

  • Figure 1: Performance of the Spice method for single-variable QCQP problems with different $\rho(t)$ functions. (a) Objective function value versus the number of iterations; (b) delta objective value versus the number of iterations.
  • Figure 2: Performance of the Spice method for separable-variable QCQP problems with different $\rho(t)$ functions. (a) Objective function value versus the number of iterations; (b) delta objective value versus the number of iterations.

Theorems & Definitions (36)

  • Lemma 2.1
  • proof
  • Lemma 2.2
  • proof
  • Lemma 2.3
  • proof
  • Remark 2.1
  • Lemma 3.1
  • proof
  • Remark 3.1
  • ...and 26 more