Shannon-and von neumann-entropy regularizations of linear and semidefinite programs
Saroj Prasad Chhatoi, Jean B Lasserre
TL;DR
The paper develops entropy-regularized formulations for linear and semidefinite programs by employing the Boltzmann-Shannon entropy for LPs and the Von Neumann entropy for SDPs. It derives equivalent unconstrained dual problems with explicit, differentiable, concave objective functions $G_\varepsilon(\lambda)$ (and $\hat{G}_\varepsilon(\lambda)$) that yield the primal solutions through simple closed-form expressions, and proves convergence of the regularized problems to the original optima as $\varepsilon\downarrow 0$. The authors highlight key differences from log-barrier methods, notably that the entropy-based duals are defined on the entire dual space and avoid barrier constraints, while still preserving coercivity and uniqueness of dual solutions. They provide asymptotic results and computational experiments demonstrating practical performance on large LPs and medium-sized SDPs, illustrating the potential for efficient unconstrained optimization in convex programs, and drawing connections to entropy-regularized optimal transport (Sinkhorn) methods. Overall, the approach offers a principled, entropy-based route to approximate and solve LPs and SDPs via explicit dual formulations, with favorable properties for large-scale problems.
Abstract
We consider the LP in standard form min {c T x\,: Ax = b; x $\ge$ 0} and inspired by $ε$-regularization in Optimal Transport, we introduce its $ε$-regularization ''min {c T x + $ε$ f (x)\,: Ax = b; x $\ge$ 0}'' via the (convex) Boltzmann-Shannon entropy f (x)\,:= i x i ln x i . We also provide a similar regularization for the semidefinite program ''min {Tr(C $\bullet$ X)\,: A(X) = b; X 0}'' but with now the so-called Von Neumann entropy, as in Quantum Optimal Transport. Importantly, both are not barriers of the LP and SDP cones respectively. We show that this problem admits an equivalent unconstrained convex problem max $λ$$\in$R m G$ε$($λ$) for an explicit concave differentiable function G$ε$ in dual variables $λ$ $\in$ R m . As $ε$ goes to zero, its optimal value converges to the optimal value of the initial LP. While it resembles the log-barrier formulation of interior point algorithm for the initial LP, it has a distinguishing advantage. Namely for fixed $λ$, G$ε$($λ$) is obtained as a minimization over the whole space x $\in$ R d (and not over x $\ge$ 0) to still obtain a nonnegative solution x($λ$) $\ge$ 0, whence an explicit form of G$ε$ very useful for its unconstrained maximization over R m .