Average-case thresholds for exact regularization of linear programs

TL;DR

This work provides an average-case analysis of exact regularization for linear programs with a Gaussian cost vector, framing exact regularization via the Gaussian measure of normal cones and their shifts under regularization. It introduces inner-cone and margin-geometric tools, including representer vectors, to derive two-sided bounds on the probability that exact regularization succeeds as a function of the regularization strength $\varepsilon$, with explicit results for canonical settings such as $Q = \mathbb{B}_{\infty}$ and quadratically regularized optimal transport. The bounds reveal dimension-dependent scalings and connect exact regularization to polyhedral geometry through the normal fan and solid-angle measures, supported by numerical experiments. A soft phase transition describes the sharp but gradual change in success probability as $\varepsilon$ varies, and the framework yields computable bounds and insights for practical LP problems and OT formulations.

Abstract

Small regularizers can preserve linear programming solutions exactly. This paper provides the first average-case analysis of exact regularization: with a standard Gaussian cost vector and fixed constraint set, bounds are established for the probability that exact regularization succeeds as a function of regularization strength. Failure is characterized via the Gaussian measure of inner cones, controlled by novel two-sided bounds on the measure of shifted cones. Results reveal dimension-dependent scaling laws and connect exact regularization of linear programs to their polyhedral geometry via the normal fan and the Gaussian (solid-angle) measure of its cones. Computable bounds are obtained in several canonical settings, including regularized optimal transport. Numerical experiments corroborate the predicted scalings and thresholds.

Figures (4)

• Figure 1: The inner cone, margin, membership condition, and representer vectors. Given a normal cone $T$ and a vector $w$, the corresponding inner cone $T \cap [T+w]$ coincides with the shifted cone $T+w$ when the membership condition $w\in T$ holds (left panel). When the membership condition fails (right panel), we can construct a "representer" vector $\tilde{w}\in T$ so that the inner cone equals $T+\tilde{w}$. The margin $M(T,w)$ is the set of vectors that exit $T$ under the translation $w$.
• Figure 1: Decomposing the margin. Consider a polyhedral cone $T$ with facets $T^1$, $T^2$ and associated inward normals $s^1, s^2$. Given a vector $w$, the margin $M(T,w)$ lies within the union of $P^1$ and $P^2$ (regions enclosed by dotted lines). The containment is tight when $w \in T$ (panel a), but not in general (panel b).
• Figure 1: Probably exact regularization on the $\mathbb{B}_\infty$ polytope. The left and right panels plot the empirically observed probability of failure of exact regularization over 20 independent trials as a colour scale over $n$-$\varepsilon$ axes, for quadratic and linear regularizations respectively. Approximate level curves for our theoretical bounds are included as green dash-dot lines.
• Figure 2: Probably exact regularization on the Birkhoff polytope. The left panel displays the empirically failure probability of exact regularization as a colour scale over $n$-$\varepsilon$ axes, with the green dash-dot line showing the approximate level set $\varepsilon n^{3/4}=2$. The right panel displays the empirical average of the regularization threshold $\overline{\varepsilon}$ against $n$ with 95% confidence intervals, together with our lower bound \ref{['eq:birkhoff_expec_bound']} as a green dash-dot line.

Theorems & Definitions (34)

• Proposition 1.1: Normal cone properties
• Proposition 1.2: Uniqueness in $P_{0}$
• Theorem 1.3: Gaussian measure of shifted cones
• Corollary 1.4: Simpler measure of shifted cones
• Theorem 1.5: Probability and normal cone geometry
• Theorem 2.1: Gaussian measure of an inner cone under the membership condition
• Corollary 2.2: Failure probability under the membership condition
• Corollary 2.3: Soft phase transition
• Lemma 2.4: Representer vectors for the margin
• Theorem 2.5: Bound via representer vectors