Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fast-Forwarding Stalling in Dykstra's Algorithm

Claudio Vestini, Idris Kempf

TL;DR

This work tackles the stalling phenomenon in Dykstra's algorithm for projecting onto intersections of polyhedral sets. It derives a closed-form expression for the stalling duration $N_{stall}$ and introduces a stall-averse fast-forwarding modification that updates the auxiliary variables in one step to skip the entire stall, while preserving the Boyle-Dykstra convergence guarantees. The proposed Algorithm $fastforward$ recomputes the state after stalling such that the iterates match those of standard Dykstra thereafter, effectively eliminating stalls in practice. Numerical experiments on a line-box intersection in $\mathbb{R}^2$ demonstrate substantially improved convergence behavior with identical eventual convergence to the true projection, highlighting practical applicability for time-critical projection tasks.

Abstract

Constrained quadratic programs and Euclidean projections are ubiquitous in engineering, arising in machine learning, estimation, control, and signal processing. Dykstra's algorithm is an iterative scheme for computing the Euclidean projection of an initial point onto the intersection of convex sets by successively projecting onto each set. Its low per-iteration computational cost makes it well-suited for solving large-scale or real-time problems where traditional optimisation routines become computationally burdensome. Despite its strong convergence guarantees, Dykstra's algorithm is known to suffer from stalling -- arbitrarily long intervals during which the primal iterates remain constant -- rendering its runtime unpredictable and severely limiting its applicability in time-critical settings. Focusing on polyhedral constraint sets, we derive a closed-form solution for the length of the stalling period once stalling is detected. This result enables a modified, stall-averse version of Dykstra's algorithm that fast-forwards the stalling period via a single, inexpensive update while preserving convergence guarantees. Numerical experiments demonstrate substantial improvements in convergence behaviour, establishing the proposed method as a practical enhancement for a broad class of projection-based algorithms.

Fast-Forwarding Stalling in Dykstra's Algorithm

TL;DR

This work tackles the stalling phenomenon in Dykstra's algorithm for projecting onto intersections of polyhedral sets. It derives a closed-form expression for the stalling duration and introduces a stall-averse fast-forwarding modification that updates the auxiliary variables in one step to skip the entire stall, while preserving the Boyle-Dykstra convergence guarantees. The proposed Algorithm recomputes the state after stalling such that the iterates match those of standard Dykstra thereafter, effectively eliminating stalls in practice. Numerical experiments on a line-box intersection in demonstrate substantially improved convergence behavior with identical eventual convergence to the true projection, highlighting practical applicability for time-critical projection tasks.

Abstract

Constrained quadratic programs and Euclidean projections are ubiquitous in engineering, arising in machine learning, estimation, control, and signal processing. Dykstra's algorithm is an iterative scheme for computing the Euclidean projection of an initial point onto the intersection of convex sets by successively projecting onto each set. Its low per-iteration computational cost makes it well-suited for solving large-scale or real-time problems where traditional optimisation routines become computationally burdensome. Despite its strong convergence guarantees, Dykstra's algorithm is known to suffer from stalling -- arbitrarily long intervals during which the primal iterates remain constant -- rendering its runtime unpredictable and severely limiting its applicability in time-critical settings. Focusing on polyhedral constraint sets, we derive a closed-form solution for the length of the stalling period once stalling is detected. This result enables a modified, stall-averse version of Dykstra's algorithm that fast-forwards the stalling period via a single, inexpensive update while preserving convergence guarantees. Numerical experiments demonstrate substantial improvements in convergence behaviour, establishing the proposed method as a practical enhancement for a broad class of projection-based algorithms.

Paper Structure

This paper contains 5 sections, 2 theorems, 10 equations, 1 figure, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Main result
  4. Numerical experiment
  5. Conclusion

Key Result

Theorem 1

Suppose that stalling starts at iteration $m$ as in Definition def:stalling and denote the active half-spaces by $\mathcal{A}\coloneqq\left\{ i\in\lbrace 0,\dots,n-1\rbrace \;\middle|\; x_{m+i}+e_{m-n+i}\not\in\mathcal{H}_{[m+i]} \right\}$. The length of the stalling period is where $\lceil\cdot\rceil$ is the ceiling operator, $k_{m-n+i}=e_{m-n+i}^T a_{[m+i]}$, and $\mathcal{S}\coloneqq\left\{ i\

Figures (1)

  • Figure 1: Dykstra's method and Algorithm \ref{['alg:fastforward']} applied to the line-box example from BAUSCHKESWISS. Top: The iterates $x_m$ of the two algorithms, with the stalling cycle of Dykstra's method highlighted in red. Middle: Squared errors $E(x_m)$, with the same stalling period highlighted. Bottom: Activity of the vertical half-space (left side of the box), where $1$ indicates that the half-space is active and $0$ indicates that it is inactive; the stalling period is highlighted in red and ends when this half-space becomes inactive.

Theorems & Definitions (3)

  • Definition 1: Stalling
  • Theorem 1: Length of Stalling Period
  • Corollary 1: Convergence of Algorithm \ref{['alg:fastforward']} DYKSTRA