Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improved algorithms and novel applications of the FrankWolfe.jl library

Mathieu Besançon, Sébastien Designolle, Jannis Halbey, Deborah Hendrych, Dominik Kuzinowicz, Sebastian Pokutta, Hannah Troppens, Daniel Viladrich Herrmannsdoerfer, Elias Wirth

TL;DR

This paper surveys the evolution of FrankWolfe.jl since its early release, detailing new FW variants (BPCG, PCG, DICG, BDICG, BCFW, ALM) and their algorithmic properties, step-size strategies, and linear minimization oracles. It highlights a modular ecosystem that supports both practitioners and algorithm designers, and discusses dynamic modifications (callbacks, pivoting) that control active-set size and computation. Empirical benchmarks across diverse problem classes (Birkhoff, nuclear-norm balls, optimal experiment design, Poisson/logistic-type problems, and spectrahedra) illustrate the relative strengths of standard FW, lazy variants, and active-set methods under varying LMO costs and problem geometries. The work emphasizes the package's dual utility as a ready-to-use toolbox and a flexible platform for developing and evaluating FW-based methods, with a view toward generalized settings and broader applications. The accompanying companion packages and Python interface further extend accessibility and interoperability in real-world workflows.

Abstract

Frank-Wolfe (FW) algorithms have emerged as an essential class of methods for constrained optimization, especially on large-scale problems. In this paper, we summarize the algorithmic design choices and progress made in the last years of the development of FrankWolfe.jl, a Julia package gathering high-performance implementations of state-of-the-art FW variants. We review key use cases of the library in the recent literature, which match its original dual purpose: first, becoming the de-facto toolbox for practitioners applying FW methods to their problem, and second, offering a modular ecosystem to algorithm designers who experiment with their own variants and implementations of algorithmic blocks. Finally, we demonstrate the performance of several FW variants on important problem classes in several experiments, which we curated in a separate repository for continuous benchmarking.

Improved algorithms and novel applications of the FrankWolfe.jl library

TL;DR

This paper surveys the evolution of FrankWolfe.jl since its early release, detailing new FW variants (BPCG, PCG, DICG, BDICG, BCFW, ALM) and their algorithmic properties, step-size strategies, and linear minimization oracles. It highlights a modular ecosystem that supports both practitioners and algorithm designers, and discusses dynamic modifications (callbacks, pivoting) that control active-set size and computation. Empirical benchmarks across diverse problem classes (Birkhoff, nuclear-norm balls, optimal experiment design, Poisson/logistic-type problems, and spectrahedra) illustrate the relative strengths of standard FW, lazy variants, and active-set methods under varying LMO costs and problem geometries. The work emphasizes the package's dual utility as a ready-to-use toolbox and a flexible platform for developing and evaluating FW-based methods, with a view toward generalized settings and broader applications. The accompanying companion packages and Python interface further extend accessibility and interoperability in real-world workflows.

Abstract

Frank-Wolfe (FW) algorithms have emerged as an essential class of methods for constrained optimization, especially on large-scale problems. In this paper, we summarize the algorithmic design choices and progress made in the last years of the development of FrankWolfe.jl, a Julia package gathering high-performance implementations of state-of-the-art FW variants. We review key use cases of the library in the recent literature, which match its original dual purpose: first, becoming the de-facto toolbox for practitioners applying FW methods to their problem, and second, offering a modular ecosystem to algorithm designers who experiment with their own variants and implementations of algorithmic blocks. Finally, we demonstrate the performance of several FW variants on important problem classes in several experiments, which we curated in a separate repository for continuous benchmarking.
Paper Structure (32 sections, 32 equations, 6 figures, 2 tables, 5 algorithms)

This paper contains 32 sections, 32 equations, 6 figures, 2 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. New Frank-Wolfe algorithms
  3. Blended Pairwise Conditional Gradients (BPCG)
  4. Pairwise Conditional Gradients (PCG)
  5. Decomposition-Invariant Conditional Gradients (DICG)
  6. Blended Decomposition-Invariant Conditional Gradients
  7. Block-Coordinate Frank-Wolfe (BCFW)
  8. Alternating Linear Minimization (ALM)
  9. A note on dual gaps
  10. New step-size strategies
  11. Open-loop step sizes
  12. Monotonic open-loop step sizes
  13. Adaptive step sizes
  14. Secant step sizes
  15. Linear Minimization Oracles
  16. ...and 17 more sections

Figures (6)

  • Figure 1: Number of solved instances for base variants.
  • Figure 2: Geometric mean of dual gaps of base variants.
  • Figure 3: Number of solved instances for lazified variants.
  • Figure 4: Geometric mean and geometric standard deviation of dual gaps for lazified variants.
  • Figure 5: Number of solved instances for active set variants with product caching.
  • ...and 1 more figures