A Relaxed Primal-Dual Hybrid Gradient Method with Line Search

TL;DR

This work addresses the sensitivity of the primal-dual hybrid gradient method (PDHG) to three user-defined parameters by introducing a parameter-free variant built from two complementary line searches. An AOI-based line search over the relaxation parameter, informed by the PDHG–Douglas–Rachford relationship, is combined with Malitsky’s line search for the proximal step sizes, yielding a relaxed PDHG with no hyperparameter tuning in the numerical experiments. The approach is validated on generalized LASSO, one- and two-dimensional total variation denoising, and a novel MRI reconstruction problem that fuses compressed sensing with partial Fourier/homodyne techniques, demonstrating competitive performance without parameter search overhead. The method enables matrix-free implementations and practical applicability to large-scale imaging problems, offering a parameter-free alternative that preserves convergence guarantees and accelerates real-world reconstructions.

Abstract

The primal-dual hybrid gradient method (PDHG) is useful for optimization problems that commonly appear in image reconstruction. A downside of PDHG is that there are typically three user-set parameters and performance of the algorithm is sensitive to their values. Toward a parameter-free algorithm, we combine two existing line searches. The first, by Malitsky et al., is over two of the step sizes in the PDHG iterations. We then use the connection between PDHG and the primal-dual form of Douglas-Rachford splitting to construct a line search over the relaxation parameter. We demonstrate the efficacy of the combined line search on multiple problems, including a novel inverse problem in magnetic resonance image reconstruction. The method presented in this manuscript is the first parameter-free variant of PDHG (across all numerical experiments, there were no changes to line search hyperparameters).

Figures (9)

• Figure 1: A comparison of the objective values from the different optimization algorithms as they solve the regularized least squares problem in \ref{['res:lasso']}.
• Figure 2: The ground truth and noisy signals used for the one-dimensional total variation denoising problem.
• Figure 3: (Left) A comparison of the objective values of different algorithms solving the one-dimensional total variation denoising problem. The line search proposed in this manuscript converges with fewer iterations than either constituent line search alone. The objective value jumps often as different step sizes are chosen. The raw objective value for each step is shown for academic purposes; in practice, one would choose the best objective value so far. The PDHG line performs better than standard PDHG. The AOI line search alone converges to a poor solution. (Right) A comparison of the best answers generated by the different optimization algorithms after 1000 iterations, plotted as the residual.
• Figure 4: The cameraman image used for the two-dimensional total variation denoising problem. On the right is the image with white noise added.
• Figure 5: (Left) A comparison of the objective values of the different optimization algorithms used for the two-dimensional total variation denoising problem. The AOI line search converged to a poor solution. The line search proposed in this manuscript took fewer iterations to converge to any given level of error and did not require searching over any parameters. The comparison against PDHG and PDHG with line search is done against their best sets of parameters. (Right) The final output of rPDHG, the line search proposed in this manuscript, solving the two dimensional TV denoising problem as in \ref{['prob:2dtv']}.