Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive projected SOR algorithms for nonnegative quadratic programming

Yuto Miyatake, Tomohiro Sogabe

TL;DR

This paper tackles efficient solution of nonnegative quadratic programs by adaptive projection via the SOR framework. It reframes PSOR as a gradient/discrete-gradient method and introduces Wolfe-condition–based APSOR updates to adapt the relaxation parameter with negligible computational overhead. Two practical extensions—(i) fixing the step size after a phase of iteration and (ii) initializing from a shifted, better-conditioned problem—address stability and stagnation in ill-conditioned or singular cases, with strong empirical performance on large-scale problems and image deblurring. The results show that APSOR variants often match or outperform PSOR with near-optimal fixed parameters, suggesting broad applicability without strong assumptions on $A$ beyond positive diagonals and SPD/SPSD structure.

Abstract

The choice of relaxation parameter in the projected successive overrelaxation (PSOR) method for nonnegative quadratic programming problems is problem-dependent. We present novel adaptive PSOR algorithms that adaptively control the relaxation parameter using the Wolfe conditions. The method and its variants can be applied to various problems without requiring additional assumptions, barring the positive semidefiniteness concerning the matrix that defines the objective function, and the cost for updating the parameter is negligible in the whole iteration. Numerical experiments show that the proposed methods often perform comparably to (or sometimes superior to) the PSOR method with a nearly optimal relaxation parameter.

Adaptive projected SOR algorithms for nonnegative quadratic programming

TL;DR

This paper tackles efficient solution of nonnegative quadratic programs by adaptive projection via the SOR framework. It reframes PSOR as a gradient/discrete-gradient method and introduces Wolfe-condition–based APSOR updates to adapt the relaxation parameter with negligible computational overhead. Two practical extensions—(i) fixing the step size after a phase of iteration and (ii) initializing from a shifted, better-conditioned problem—address stability and stagnation in ill-conditioned or singular cases, with strong empirical performance on large-scale problems and image deblurring. The results show that APSOR variants often match or outperform PSOR with near-optimal fixed parameters, suggesting broad applicability without strong assumptions on beyond positive diagonals and SPD/SPSD structure.

Abstract

The choice of relaxation parameter in the projected successive overrelaxation (PSOR) method for nonnegative quadratic programming problems is problem-dependent. We present novel adaptive PSOR algorithms that adaptively control the relaxation parameter using the Wolfe conditions. The method and its variants can be applied to various problems without requiring additional assumptions, barring the positive semidefiniteness concerning the matrix that defines the objective function, and the cost for updating the parameter is negligible in the whole iteration. Numerical experiments show that the proposed methods often perform comparably to (or sometimes superior to) the PSOR method with a nearly optimal relaxation parameter.
Paper Structure (13 sections, 7 theorems, 46 equations, 8 figures, 5 algorithms)

This paper contains 13 sections, 7 theorems, 46 equations, 8 figures, 5 algorithms.

Table of Contents

  1. Introduction
  2. Projected SOR method
  3. Adaptive projected SOR methods
  4. Geometric interpretation of the SOR method
  5. Adaptive projected SOR (APSOR) methods
  6. Extensions of APSOR method
  7. Numerical experiments
  8. Toy problems with non-singular matrix $A$
  9. Toy problems with singular matrix $A$
  10. Image deblurring
  11. Concluding remarks
  12. Proof of Theorem \ref{['thm:psor_spd']}
  13. Codes for generating toy problems

Key Result

theorem 1

Assume that $A$ is symmetric positive definite. Let $\bm x^\ast$ be the unique solution of nqp. The sequence of iterates generated by the PSOR (Algorithm algo:sor) converges to $\bm x^\ast$, i.e. $\bm x^{(k)} \to \bm x^\ast$ as $k\to \infty$ if and only if $\omega\in (0,2)$.

Figures (8)

  • Figure 1: Test problem (the condition number is $10$). Convergence behaviour of the proposed algorithms in comparison with the PSOR method with several (fixed) relaxation parameters. Bottom figure shows how the relaxation parameter of the proposed algorithms changes during the iterations.
  • Figure 2: Test problem (the condition number is $10^4$). Convergence behaviour of the proposed algorithms in comparison with the PSOR method with several (fixed) relaxation parameters. The fastest PSOR is with $\omega = 1.8$. Bottom figure shows how the relaxation parameter of the proposed algorithms changes during the iterations.
  • Figure 3: Test problem (the condition number is $10^7$). Convergence behaviour of the proposed algorithms in comparison with the PSOR method with several (fixed) relaxation parameters. The fastest PSOR is with $\omega = 1.9$. Bottom figure shows how the relaxation parameter of the proposed algorithms changes during the iterations.
  • Figure 4: Test problem (the condition number is $10^{10}$). Convergence behaviour of the proposed algorithms in comparison with the PSOR method with several (fixed) relaxation parameters. The fastest PSOR is with $\omega = 1.9$. The results of the PSOR with $\omega= 1.85$ and $1.95$ exhibit similar behaviour. Bottom figure shows how the relaxation parameter of the proposed algorithms changes during the iterations.
  • Figure 5: (Left) Convergence bahaviour of Algorithm \ref{['algo:asor2']} applied to the first example in Section \ref{['subsec:num_singular']} in comparison with the PSOR method with several (fixed) relaxation parameters. PSOR with $\omega = 1.0$ spent 11,837 iterations to convergence. (Right) Relaxation parameter of Algorithm \ref{['algo:asor2']} selected at each iteration.
  • ...and 3 more figures

Theorems & Definitions (11)

  • theorem 1: e.g. cr71a
  • remark thmcounterremark
  • theorem 2: lu91
  • remark thmcounterremark
  • theorem 3: ms18
  • lemma thmcounterlemma
  • remark thmcounterremark
  • lemma thmcounterlemma
  • theorem 4
  • remark thmcounterremark
  • ...and 1 more