Algorithmic approaches to avoiding bad local minima in nonconvex inconsistent feasibility

TL;DR

This work investigates projection-based methods for nonconvex inconsistent feasibility, focusing on how algorithmic tuning can mitigate bad local minima rather than relying on regularization. It analyzes cyclic projections, cyclic relaxed Douglas–Rachford (CDRλ), and relaxed Douglas–Rachford on the product space for the 3D ARPES orbital tomography problem, formulating a multi-set feasibility model with SYM, SR, SUPP, LF, and M constraints. The authors establish local linear convergence under epsilon-super-regularity, characterize fixed-point sets, and show through simulated and experimental ARPES data that cyclic projections is fast but DR methods, particularly the product-space DR, can effectively filter out poor fixed points and move toward globally meaningful minima. Practically, they recommend a CP initialization followed by DRλ on the product-space formulation with a high, stable λ to reduce gaps between constraint sets, offering a robust strategy for high-quality reconstructions in phase retrieval-like imaging tasks.

Abstract

The main challenge of nonconvex optimization is to find a global optimum, or at least to avoid ``bad'' local minima and meaningless stationary points. We study here the extent to which algorithms, as opposed to optimization models and regularization, can be tuned to accomplish this goal. The model we consider is a nonconvex, inconsistent feasibility problem with many local minima, where these are points at which the gaps between the sets are smallest on neighborhoods of these points. The algorithms that we compare are all projection-based algorithms, specifically cyclic projections, the cyclic relaxed Douglas-Rachford algorithm, and relaxed Douglas-Rachford splitting on the product space. The local convergence and fixed points of these algorithms have already been characterized in pervious theoretical studies. We demonstrate the theory for these algorithms in the context of orbital tomographic imaging from angle-resolved photon emission spectroscopy (ARPES) measurements, both synthetically generated and experimental. Our results show that, while the cyclic projections and cyclic relaxed Douglas-Rachford algorithms generally converge the fastest, the method of relaxed Douglas-Rachford splitting on the product space does move away from bad local minima of the other two algorithms, settling eventually on clusters of local minima corresponding to globally optimal critical points.

Figures (7)

• Figure 1: a) Data obtained from ARPES; b) Fourier amplitude of the exact solution; c) Fourier amplitude of reconstruction at smallest gap of the CDR$\lambda$ with $\lambda=0.7$; d) The exact solution; e) Reconstruction at smallest gap of the CDR$\lambda$ with $\lambda=0.7$.
• Figure 2: (a) Five "typical" convergence plots of the cyclic relaxed Douglash-Rachford and cyclic projection algorithms. (b) The relationship between gap and error at (approximate) fixed points of the two algorithms.
• Figure 3: CP and CDR$\lambda$, $\lambda=0.7, 0.3$ with 100 trials for each method. (a) Boxplots showing gap variation. (b) Histogram of the gap.
• Figure 4: DR$\lambda$. Convergence plots of changes (monitor$_1$, left column) and gaps (monitor$_2$, right column) for the algorithm with two values of $\lambda$. Each $\lambda$ was run with 50 different starting points.
• Figure 5: (a) Comparison of the efficiency between cyclic projection and relaxed Douglas-Rachford with $\lambda = 0.5$ and $\lambda = 0.7$. (b) Stabilization of approximate fixed points of relaxed Douglas-Rachford, with $\lambda = 0.7$, starting from random initialization, and the approximate fixed points of the relaxed Douglas-Rachford with $\lambda = 0.5$.

Theorems & Definitions (21)

• Lemma 1
• proof
• Lemma 2
• proof
• Lemma 3
• proof
• Lemma 4
• proof
• Lemma 5
• proof