Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fast projection onto the intersection of simplex and singly linear constraint and its generalized Jacobian

Weimi Zhou, Yong-Jin Liu

TL;DR

This work addresses the projection of a vector onto the intersection of a simplex and a single linear inequality. By formulating the problem via Lagrangian duality, the authors reduce it to solving a univariate nonsmooth equation \psi(\\sigma)=0 and develop two fast algorithms: LRSA, which uses a modified secant method with a bracketing phase, and SSN, which leverages semismooth Newton iterations with an explicit generalized differential. Numerical experiments show LRSA and SSN substantially outperform the general-purpose solver Gurobi, especially on large-scale instances, and degenerate cases where the structure can be exploited. Additionally, the paper derives explicit formulas for the generalized HS-Jacobian of the projection \Pi_{\\mathcal{C}}(\\bm{y}), enabling efficient design of second-order nonsmooth methods. Together, these contributions advance fast, structure-exploiting projection methods for distributionally robust optimization and related convex-constrained problems.

Abstract

Solving the distributional worst-case in the distributionally robust optimization problem is equivalent to finding the projection onto the intersection of simplex and singly linear inequality constraint. This projection is a key component in the design of efficient first-order algorithms. This paper focuses on developing efficient algorithms for computing the projection onto the intersection of simplex and singly linear inequality constraint. Based on the Lagrangian duality theory, the studied projection can be obtained by solving a univariate nonsmooth equation. We employ an algorithm called LRSA, which leverages the Lagrangian duality approach and the secant method to compute this projection. In this algorithm, a modified secant method is specifically designed to solve the piecewise linear equation. Additionally, due to semismoothness of the resulting equation, the semismooth Newton (SSN) method is a natural choice for solving it. Numerical experiments demonstrate that LRSA outperforms SSN algorithm and the state-of-the-art optimization solver called Gurobi. Moreover, we derive explicit formulas for the generalized HS-Jacobian of the projection, which are essential for designing second-order nonsmooth Newton algorithms.

Fast projection onto the intersection of simplex and singly linear constraint and its generalized Jacobian

TL;DR

This work addresses the projection of a vector onto the intersection of a simplex and a single linear inequality. By formulating the problem via Lagrangian duality, the authors reduce it to solving a univariate nonsmooth equation \psi(\\sigma)=0 and develop two fast algorithms: LRSA, which uses a modified secant method with a bracketing phase, and SSN, which leverages semismooth Newton iterations with an explicit generalized differential. Numerical experiments show LRSA and SSN substantially outperform the general-purpose solver Gurobi, especially on large-scale instances, and degenerate cases where the structure can be exploited. Additionally, the paper derives explicit formulas for the generalized HS-Jacobian of the projection \Pi_{\\mathcal{C}}(\\bm{y}), enabling efficient design of second-order nonsmooth methods. Together, these contributions advance fast, structure-exploiting projection methods for distributionally robust optimization and related convex-constrained problems.

Abstract

Solving the distributional worst-case in the distributionally robust optimization problem is equivalent to finding the projection onto the intersection of simplex and singly linear inequality constraint. This projection is a key component in the design of efficient first-order algorithms. This paper focuses on developing efficient algorithms for computing the projection onto the intersection of simplex and singly linear inequality constraint. Based on the Lagrangian duality theory, the studied projection can be obtained by solving a univariate nonsmooth equation. We employ an algorithm called LRSA, which leverages the Lagrangian duality approach and the secant method to compute this projection. In this algorithm, a modified secant method is specifically designed to solve the piecewise linear equation. Additionally, due to semismoothness of the resulting equation, the semismooth Newton (SSN) method is a natural choice for solving it. Numerical experiments demonstrate that LRSA outperforms SSN algorithm and the state-of-the-art optimization solver called Gurobi. Moreover, we derive explicit formulas for the generalized HS-Jacobian of the projection, which are essential for designing second-order nonsmooth Newton algorithms.
Paper Structure (12 sections, 10 theorems, 62 equations, 3 tables, 3 algorithms)

This paper contains 12 sections, 10 theorems, 62 equations, 3 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. The projection onto the simplex
  3. Efficient algorithms based on Lagrangian duality method
  4. Lagrangian duality method
  5. An algorithm based on Lagrangian duality approach and secant method
  6. An algorithm based on semismooth Newton method
  7. The generalized differential
  8. Algorithm description
  9. Numerical experiments
  10. The generalized Jacobian of the projection $\Pi_{\mathcal{C}}(\cdot)$
  11. Conlusions
  12. Acknowledgments

Key Result

Proposition 2.1

The following properties hold:

Theorems & Definitions (18)

  • Proposition 2.1
  • Proposition 2.2
  • Proposition 3.1
  • proof
  • Proposition 3.2
  • proof
  • Theorem 3.3
  • Theorem 3.4
  • Remark 3.5
  • Theorem 3.6
  • ...and 8 more