An efficient active-set method with applications to sparse approximations and risk minimization
Spyridon Pougkakiotis, Jacek Gondzio, Dionysis Kalogerias
TL;DR
This work tackles convex quadratic programs with piecewise-linear terms and box constraints by marrying a primal-dual proximal method of multipliers (PMM) with a semismooth Newton (SSN) inner solver, yielding an active-set framework. By dualizing nonsmooth terms and introducing continuously differentiable PMM subproblems, the authors enable efficient SSN solves whose linear systems are tackled via novel, problem-agnostic Krylov preconditioners that remain well-conditioned as PMM penalties grow. They establish global convergence under mild feasibility assumptions and local (super)linear convergence under standard conditions, with a linear-algebra kernel designed to be memory- and compute-efficient. Extensive numerical experiments across risk-averse portfolio optimization (CVaR and MAD), L1/L2 PDE-constrained optimization, penalized quantile regression, and linear SVM demonstrate robust, scalable performance and competitive, often superior, efficiency relative to IP-PMM and first-order baselines, highlighting the practical impact of the approach for large-scale, nonsmooth convex optimization problems.
Abstract
In this paper we present an efficient active-set method for the solution of convex quadratic programming problems with general piecewise-linear terms in the objective, with applications to sparse approximations and risk-minimization. The algorithm is derived by combining a proximal method of multipliers (PMM) with a standard semismooth Newton method (SSN), and is shown to be globally convergent under minimal assumptions. Further local linear (and potentially superlinear) convergence is shown under standard additional conditions. The major computational bottleneck of the proposed approach arises from the solution of the associated SSN linear systems. These are solved using a Krylov-subspace method, accelerated by certain novel general-purpose preconditioners which are shown to be optimal with respect to the proximal penalty parameters. The preconditioners are easy to store and invert, since they exploit the structure of the nonsmooth terms appearing in the problem's objective to significantly reduce their memory requirements. We showcase the efficiency, robustness, and scalability of the proposed solver on a variety of problems arising in risk-averse portfolio selection, $L^1$-regularized partial differential equation constrained optimization, quantile regression, and binary classification via linear support vector machines. We provide computational evidence, on real-world datasets, to demonstrate the ability of the solver to efficiently and competitively handle a diverse set of medium- and large-scale optimization instances.