Homotopy Methods for Convex Optimization

TL;DR

The paper addresses solving general convex optimization problems by deforming the feasible set from an easy start to the target set and tracking the optimal solution along a path. The approach replaces traditional central-path iterations with a continuous path defined by a differential equation that governs the evolution of stationary points. It establishes existence and uniqueness of this path under mild regularity conditions and demonstrates applicability to semidefinite and hyperbolic programs, as well as single-constraint convex problems, with smoothing techniques for real-zero polynomials to ensure regularity. Numerical experiments show substantial speedups in hyperbolic programming and highlight the method's potential as a practical alternative to interior-point methods across several convex problem classes.

Abstract

Convex optimization encompasses a wide range of optimization problems that contain many efficiently solvable subclasses. Interior point methods are currently the state-of-the-art approach for solving such problems, particularly effective for classes like semidefinite programming, quadratic programming, and geometric programming. However, their success hinges on the construction of self-concordant barrier functions for feasible sets. In this work, we investigate and develop a homotopy-based approach to solve convex optimization problems. While homotopy methods have been considered in optimization before, their potential for general convex programs remains underexplored. This approach gradually transforms the feasible set of a trivial optimization problem into the target one while tracking solutions by solving a differential equation, in contrast to traditional central path methods. We establish a criterion that ensures that the homotopy method correctly solves the optimization problem and prove the existence of such homotopies for several important classes, including semidefinite and hyperbolic programs. Furthermore, we demonstrate that our approach numerically outperforms state-of-the-art methods in hyperbolic programming, highlighting its practical advantages.

Figures (8)

• Figure 1: Visualization of the method presented in this paper. To solve the convex optimization problem on the right (i.e. maximizing $\mathbf{x} \mapsto \mathbf{b}^t \mathbf{x}$ over a convex set), one instead solves a trivial convex optimization problem (left) and tracks the path of optimal solutions via solving a differential equation.
• Figure 2: Examples of the homotopy within the space of real zero polynomials for different smoothing parameters. For $\varepsilon = 0$, the homotopy leads to non-smooth convex sets (see for example $t = 0.5$ and $t = 0.6$), as soon as $\varepsilon > 0$, the roots of multiplicity $2$ become roots of multiplicity $1$.
• Figure 3: Optimizing the linear function $f(x_1, x_2) = x_2$ over a (non-smooth) spectrahedron (blue). The three paths are obtained by using the the homotopies $\hat{p}_t$ (left, blue), $p_t^{[1]}$ (middle, red) and $p_{t}^{[2]}$ (right, green). While only $p^{[2]}$ is a smooth homotopy, numerically all paths lead to a correct solution. The blue path on the left has two singularities, the red path in the middle has only one singularity and the green path on the right has no singularity.
• Figure 4: Homotopy of the convex optimization problem presented in \ref{['ex:convexOptimization problem']}.
• Figure 5: Runtimes of the homotopy method (blue) vs. the SDP solver (orange) applied to a random linear functional and $\mathcal{R}(p_k)$ for different dimensions $n$. For the SDP solver, we use the spectrahedral representation of $\mathcal{R}(p_k)$ from Br14. The SDP solver only reaches a conclusion for $n \leqslant 5$ and $n \leqslant 6$, respectively due to the growth of the matrix sizes (see \ref{['tab:matrixSize_elementary_symmetric']}).

Theorems & Definitions (29)

• Definition 1: Smooth description
• Definition 2: Homotopy of smooth descritions
• Remark 3
• Lemma 4
• proof
• Lemma 5
• proof
• Theorem 6
• proof
• Remark 7