Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Strongly Convex Maximization via the Frank-Wolfe Algorithm with the Kurdyka-Łojasiewicz Inequality

Fatih Selim Aktas, Christian Kroer

TL;DR

The work analyzes the greedy Frank-Wolfe algorithm with unit steps for convex maximization over a compact set when the objective is strongly convex and smooth, and the KL property holds. By reformulating the problem as a gradient-like descent sequence and leveraging KL theory, the authors establish global last-iterate convergence to stationary/critical points and provide problem-dependent convergence rates. They demonstrate the framework on three domains—sparse PCA, reweighted $\ell_1$ minimization for compressed sensing, and Max-Cut SDP—showing that the KL property (via semialgebraic structure) yields convergent, scalable algorithms with practical performance gains. The results offer a unified convergence analysis for GFW in non-polyhedral settings and new convergent variants for important sparse and SDP-relaxation problems.

Abstract

We study the convergence properties of the 'greedy' Frank-Wolfe algorithm with a unit step size, for a convex maximization problem over a compact set. We assume the function satisfies smoothness and strong convexity. These assumptions together with the Kurdyka-Łojasiewicz (KL) property, allow us to derive global asymptotic convergence for the sequence generated by the algorithm. Furthermore, we also derive a convergence rate that depends on the geometric properties of the problem. To illustrate the implications of the convergence result obtained, we prove a new convergence result for a sparse principal component analysis algorithm, propose a convergent reweighted $\ell_1$ minimization algorithm for compressed sensing, and design a new algorithm for the semidefinite relaxation of the Max-Cut problem.

Strongly Convex Maximization via the Frank-Wolfe Algorithm with the Kurdyka-Łojasiewicz Inequality

TL;DR

The work analyzes the greedy Frank-Wolfe algorithm with unit steps for convex maximization over a compact set when the objective is strongly convex and smooth, and the KL property holds. By reformulating the problem as a gradient-like descent sequence and leveraging KL theory, the authors establish global last-iterate convergence to stationary/critical points and provide problem-dependent convergence rates. They demonstrate the framework on three domains—sparse PCA, reweighted minimization for compressed sensing, and Max-Cut SDP—showing that the KL property (via semialgebraic structure) yields convergent, scalable algorithms with practical performance gains. The results offer a unified convergence analysis for GFW in non-polyhedral settings and new convergent variants for important sparse and SDP-relaxation problems.

Abstract

We study the convergence properties of the 'greedy' Frank-Wolfe algorithm with a unit step size, for a convex maximization problem over a compact set. We assume the function satisfies smoothness and strong convexity. These assumptions together with the Kurdyka-Łojasiewicz (KL) property, allow us to derive global asymptotic convergence for the sequence generated by the algorithm. Furthermore, we also derive a convergence rate that depends on the geometric properties of the problem. To illustrate the implications of the convergence result obtained, we prove a new convergence result for a sparse principal component analysis algorithm, propose a convergent reweighted minimization algorithm for compressed sensing, and design a new algorithm for the semidefinite relaxation of the Max-Cut problem.
Paper Structure (12 sections, 10 theorems, 39 equations, 5 figures, 1 table, 3 algorithms)

This paper contains 12 sections, 10 theorems, 39 equations, 5 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Review of Existing Results
  3. Frank-Wolfe
  4. Maximizing a Strongly Convex Function with Greedy Frank-Wolfe
  5. Gradient-Like Descent Sequences
  6. Global Convergence of GFW under the KL property
  7. Applications
  8. Reweighted Optimization for Minimizing the l0 Norm
  9. Sparse PCA Lower Bound Approach
  10. Simple Parallel Algorithm for the Max-Cut SDP Formulation
  11. Conclusion
  12. Convergence to a Strict Local Minimum over Polyhedra

Key Result

Proposition 2.1

Let $g: {\mathbb R}^d \mapsto {\mathbb R}$ be a convex function and $S \subset {\mathbb R}^d$ be an arbitrary set. Let $\textbf{conv}(S)$ denote its convex hull. Then

Figures (5)

  • Figure 1: Illustration of a single iteration of the GFW algorithm on two nonconvex sets. The objective is strictly increasing along the path of the maximizer of the gradient inner product, thus ignoring the points in between, which may or may not be in the feasible set.
  • Figure 2: Empirical comparison of the RWL1, RWL1 Prox, and RWL1 Split algorithms on sparse signal recovery problems for changing cardinality level
  • Figure 3: Empirical comparison of the RWL1, RWL1 Prox, and RWL1 Split algorithms on noisy sparse signal recovery problems for changing cardinality level
  • Figure 4: Empirical comparison of various algorithms for solving the Max-Cut SDP Relaxation. The figure on the left shows the objective value against the wall time. GFW and GFW & RTR algorithms overlap until $t \approx 7$ seconds. The figure on the right provides a zoomed-in view, focusing on the final $5$ seconds.
  • Figure 5: Empirical performance of the GFW algorithm for varying shifting parameter $\sigma$ on the Max-Cut SDP relaxation. The figure on the left shows the objective value against the wall time. The figure on the right provides a zoomed-in view, focusing on the final $5$ seconds.

Theorems & Definitions (17)

  • Proposition 2.1
  • Proposition 2.2
  • proof
  • Lemma 2.3
  • proof
  • Theorem 2.4
  • proof
  • Definition 2.5
  • Definition 2.6
  • Theorem 2.7
  • ...and 7 more