Soft-Radial Projection for Constrained End-to-End Learning

TL;DR

This work tackles the gradient-saturation problem of enforcing hard convex constraints in end-to-end learning by introducing Soft-Radial Projection, a differentiable radial map that sends unconstrained predictions into the interior of a convex feasible set. It provides rigorous guarantees: a homeomorphism with invertible Jacobian a.e., equivalence of constrained and unconstrained optimal values, and preservation of universal approximation. The approach yields superior convergence and solution quality compared to projection- and optimization-based baselines in tasks like portfolio optimization and ride-sharing, while avoiding iterative solvers. Practically, it enables safe, constraint-satisfying predictions in safety-critical systems without sacrificing gradient flow or expressivity.

Abstract

Integrating hard constraints into deep learning is essential for safety-critical systems. Yet existing constructive layers that project predictions onto constraint boundaries face a fundamental bottleneck: gradient saturation. By collapsing exterior points onto lower-dimensional surfaces, standard orthogonal projections induce rank-deficient Jacobians, which nullify gradients orthogonal to active constraints and hinder optimization. We introduce Soft-Radial Projection, a differentiable reparameterization layer that circumvents this issue through a radial mapping from Euclidean space into the interior of the feasible set. This construction guarantees strict feasibility while preserving a full-rank Jacobian almost everywhere, thereby preventing the optimization stalls typical of boundary-based methods. We theoretically prove that the architecture retains the universal approximation property and empirically show improved convergence behavior and solution quality over state-of-the-art optimization- and projection-based baselines.

Figures (6)

• Figure 1: Impact of projection geometry on optimization. Comparison of (a) Orthogonal and (b) Soft-Radial Projection on a 2D constrained task (target $\times$). Main plots visualize the trajectory of the unconstrained candidate ($\circ$) versus the feasible decision ($+$). Insets show the coordinate grid warping (A) and training loss (B). Note that soft-radial projection prevents gradient saturation by maintaining descent signals for infeasible inputs.
• Figure 2: Geometric intuition. A ray from anchor $u_0$ through input $u$ intersects the boundary $\partial\mathcal{C}$ at the hard radial projection $q(u)$. Our soft-radial mapping $p(u)$ strictly enforces feasibility by smoothly interpolating along the segment $[u_0, q(u)]$ via the radial weight $r$.
• Figure 3: Soft-radial projection $p(u)$ with rational radial contraction \ref{['eq:rational']}.
• Figure 4: Soft-radial projection $p(u)$ with exponential radial contraction \ref{['eq:exponential']}.
• Figure 5: Soft-radial projection $p(u)$ with hyperbolic radial contraction \ref{['eq:hyperbolic']}.

Theorems & Definitions (35)

• Definition 2.1: Soft-Radial Projection
• Remark 2.2
• Lemma 2.2
• Definition 2.4: Boundary distance
• Definition 2.5: Scalar projection map
• Lemma 2.5
• Theorem 2.6
• Theorem 2.7
• Theorem 3.1
• Proposition 3.1