Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Parallel Newton methods for the continuous quadratic knapsack problem: A Jacobi and Gauss-Seidel tale

Leonardo D. Secchin, Paulo J. S. Silva

Abstract

The continuous quadratic knapsack (CQK) problem involves minimizing a diagonal convex quadratic function subject to box constraints and a single linear equality constraint. It has numerous applications in resource allocation, multicommodity flow, machine learning, and classical optimization tasks such as Lagrangian relaxation and quasi-Newton updates. In this work, we revisit the semismooth Newton method introduced by Cominetti, Mascarenhas, and Silva. We demonstrate that the method can be significantly improved in two directions. First, for projections onto the simplex or the $\ell_1$-ball, it can incorporate Condat's highly effective initial multiplier guess. Second, it can serve as a flexible foundation for CQK algorithms, allowing for different parallel variants tailored to exploit CPU and GPU computational models. These improvements are implemented in the open-source Julia package \texttt{NewtonCQK.jl}. We present extensive numerical tests comparing this implementation with other state-of-the-art solvers, demonstrating its superior efficiency and scalability.

Parallel Newton methods for the continuous quadratic knapsack problem: A Jacobi and Gauss-Seidel tale

Abstract

The continuous quadratic knapsack (CQK) problem involves minimizing a diagonal convex quadratic function subject to box constraints and a single linear equality constraint. It has numerous applications in resource allocation, multicommodity flow, machine learning, and classical optimization tasks such as Lagrangian relaxation and quasi-Newton updates. In this work, we revisit the semismooth Newton method introduced by Cominetti, Mascarenhas, and Silva. We demonstrate that the method can be significantly improved in two directions. First, for projections onto the simplex or the -ball, it can incorporate Condat's highly effective initial multiplier guess. Second, it can serve as a flexible foundation for CQK algorithms, allowing for different parallel variants tailored to exploit CPU and GPU computational models. These improvements are implemented in the open-source Julia package \texttt{NewtonCQK.jl}. We present extensive numerical tests comparing this implementation with other state-of-the-art solvers, demonstrating its superior efficiency and scalability.
Paper Structure (17 sections, 2 theorems, 20 equations, 8 figures, 4 tables, 4 algorithms)

This paper contains 17 sections, 2 theorems, 20 equations, 8 figures, 4 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Notation
  3. Sequential methods for CQK
  4. Newton methods for CQK
  5. Variable fixing
  6. A specialized Newton method to project onto a simplex
  7. Warm start and variable fixing
  8. Projection onto a $\ell_1$ ball
  9. Parallelizing CQK methods
  10. Numerical tests
  11. Tests with general quadratic knapsack problems
  12. Randomly generate instances
  13. Support vector machines
  14. Projection onto simplex and $\ell_1$ ball
  15. Randomly generate instances
  16. ...and 2 more sections

Key Result

Theorem 3.1

Consider Algorithm alg:purenewtonCQK applied to PDelta. Suppose that $\lambda_0\geq \underline y=\min_i\{-y_i\}$ is not a zero of $\varphi(\lambda) - r$. Then

Figures (8)

  • Figure 1: (a) Lateral slopes of $\max\{l_i,\min\{u_i, (b_i\lambda + a_i)/d_i\}\}$ at breakpoints. (b) A typical plot of $\varphi(\cdot)-r$.
  • Figure 2: Speedups of parallel Algorithm \ref{['alg:purenewtonCQK']} with variable fixing relative to the sequential version on random instances.
  • Figure 3: Relative performance of Algorithm \ref{['alg:purenewtonCQK']} from NewtonCQK.jl and CMS using variable fixing and warm starts to solve \ref{['svmsubprob']} along the SPG iterations. The baseline (horizontal blue line at 1) is the performance of the NewtonCQK.jl without warm start.
  • Figure 4: Speedups of parallel Algorithm \ref{['alg:purenewtonCQK']} for solving \ref{['svmsubprob']} in relation to the sequential algorithm. All algorithms were executed using the current SPG iterate as warm start.
  • Figure 5: Speedups of Algorithm \ref{['alg:purenewtonsimplex']} implemented in NewtonCQK.jl (CPU versions returning dense and sparse solutions), Dai and Chen's (sparse solution) and Condat's (dense solution) algorithms on \ref{['PDelta']}, instances of type 1. The base algorithm is of Condat, which is represented by the horizontal line.
  • ...and 3 more figures

Theorems & Definitions (3)

  • Theorem 3.1
  • proof
  • Lemma 3.1