Rank-one convexification for quadratic optimization problems with step function penalties

TL;DR

This work develops a rank-one convexification framework for quadratic problems with step-function penalties, reframing them as mixed-integer quadratic problems with sign-indicator constraints. It derives an explicit description of the closure of the convex hull for rank-one quadratic epigraphs and constructs copositive and SDP relaxations for a general extended set, enabling tractable, strong convexifications. The methods are applied to support vector machines with 0--1 loss, yielding robust estimators that handle outliers and anomalies while providing improved bounds and computational efficiency over traditional big-M MIO formulations. Computational experiments on synthetic and real data demonstrate that the proposed relaxations offer tighter bounds and competitive prediction performance, especially in high-dimensional, noisy settings where standard SVM approaches struggle. Overall, the paper provides a principled convexification approach that enhances both the theoretical tightness of the relaxations and the practical performance of robust SVMs.

Abstract

We investigate convexification for convex quadratic optimization with step function penalties. Such problems can be cast as mixed-integer quadratic optimization problems, where binary variables are used to encode the non-convex step function. First, we derive the convex hull for the epigraph of a quadratic function defined by a rank-one matrix and step function penalties. Using this rank-one convexification, we develop copositive and semi-definite relaxations for general convex quadratic functions. Leveraging these findings, we construct convex formulations to the support vector machine problem with 0--1 loss and show that they yield robust estimators in settings with anomalies and outliers.

Figures (3)

• Figure 1: Non-convex robust losses approximating the 0-1 loss, as a function of $x=y_i\bm{a_i^\top w}$. Top row: loss functions from the literature: the normalized sigmoid loss mason1999boosting, the $\psi$-learning loss shen2003psi and the ramp loss wu2007robust. Bottom row: the derived $\phi$ loss in Proposition \ref{['prop:loss']} for different values of hyperparameters $d$ (with $\lambda=1$) . By solving the conic optimization problem \ref{['eq:SVMReform1']}, $\bm{d}$ is chosen automatically to ensure the convexity of the ensuing learning problem.
• Figure 2: Sample synthetic datasets with $p=2$ and $n=100$.
• Figure 3: Distribution of out-of-sample misclassification for data with clustered outliers and $\sigma=0.2$, as a function of the number of datapoints $n$. In instances with small $n$ (top row), the hinge estimator has a probability of breaking down, resulting in out-of-sample misclassifications above 50%; the conic loss reduces the average misclassification rate by an order-of-magnitude. Moreover, the conic estimator performs consistently well in all settings.

Theorems & Definitions (31)

• Proposition 1: atamturk2019rankFrangioni2006
• Proposition 2: Validity
• proof
• Proposition 3
• proof
• Theorem 1
• proof
• Remark 1: Case $n=1$
• Remark 2: General case
• Proposition 4