Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Accelerated primal dual fixed point algorithm

Ya-Nan Zhu

TL;DR

The paper tackles the structured convex optimization problem $F(x)=f(x)+g(Bx)$ where $f$ has an $L_f$-Lipschitz gradient and $g$ is convex l.s.c. and may be non-smooth. It introduces Accelerated Primal-Dual Fixed Point (APDFP), which injects Nesterov-style acceleration into the PDFP framework while preserving fully decoupled iterations, enabling efficient handling of a general linear operator $B$. The authors prove convergence of APDFP with a bound on the partial primal-dual gap that scales as $\mathcal{O}(1/k^2)$ with respect to $L_f$, under standard parameter choices, and demonstrate robustness to $\lambda$ and minimal tuning. Empirically, APDFP outperforms its non-accelerated counterparts and competes favorably with other accelerated schemes on graph-guided logistic regression and 2D CT reconstruction, achieving faster convergence and higher quality metrics in fewer iterations. The work thus provides a scalable, parameter-tolerant method for large-scale composite problems where $B$ is a general linear operator, with clear practical impact in imaging and data analysis.

Abstract

This work proposes an Accelerated Primal-Dual Fixed-Point (APDFP) method that employs Nesterov type acceleration to solve composite problems of the form min f(x) + g(Bx), where g is nonsmooth and B is a linear operator. The APDFP features fully decoupled iterations and can be regarded as a generalization of Nesterov's accelerated gradient in the setting where B can be a non-identity matrix. Theoretically, we improve the convergence rate of the partial primal-dual gap with respect to the Lipschitz constant of the gradient of f from O(1/k) to O(1/k^2). Numerical experiments on graph-guided logistic regression and CT image reconstruction are conducted to validate the correctness and demonstrate the efficiency of the proposed method.

Accelerated primal dual fixed point algorithm

TL;DR

The paper tackles the structured convex optimization problem where has an -Lipschitz gradient and is convex l.s.c. and may be non-smooth. It introduces Accelerated Primal-Dual Fixed Point (APDFP), which injects Nesterov-style acceleration into the PDFP framework while preserving fully decoupled iterations, enabling efficient handling of a general linear operator . The authors prove convergence of APDFP with a bound on the partial primal-dual gap that scales as with respect to , under standard parameter choices, and demonstrate robustness to and minimal tuning. Empirically, APDFP outperforms its non-accelerated counterparts and competes favorably with other accelerated schemes on graph-guided logistic regression and 2D CT reconstruction, achieving faster convergence and higher quality metrics in fewer iterations. The work thus provides a scalable, parameter-tolerant method for large-scale composite problems where is a general linear operator, with clear practical impact in imaging and data analysis.

Abstract

This work proposes an Accelerated Primal-Dual Fixed-Point (APDFP) method that employs Nesterov type acceleration to solve composite problems of the form min f(x) + g(Bx), where g is nonsmooth and B is a linear operator. The APDFP features fully decoupled iterations and can be regarded as a generalization of Nesterov's accelerated gradient in the setting where B can be a non-identity matrix. Theoretically, we improve the convergence rate of the partial primal-dual gap with respect to the Lipschitz constant of the gradient of f from O(1/k) to O(1/k^2). Numerical experiments on graph-guided logistic regression and CT image reconstruction are conducted to validate the correctness and demonstrate the efficiency of the proposed method.

Paper Structure

This paper contains 15 sections, 5 theorems, 45 equations, 6 figures, 1 table.

Table of Contents

  1. Introduction.
  2. $B = I$.
  3. $B \not = I$.
  4. Primal Dual Hybrid Gradient.
  5. Alternating Direction Method of Multipliers.
  6. Primal Dual Fixed Point.
  7. Organization of the paper.
  8. Algorithm.
  9. Convergence.
  10. Numerical Experiments
  11. Graph-Guided Logistic regression
  12. 2D CT reconstruction.
  13. Effect of $\lambda$.
  14. Conclusions.
  15. Appendix

Key Result

Theorem 3.2

\newlabelthm10 Suppose the function $f$ is $L_f$ smooth convex function and $g$ is convex Lipchitz continuous. Choose the parameter $0 < \lambda \leq \frac{1}{\rho_{\max}(BB^T)}$, and select a sequence of parameters $\gamma_k$ in Algorithm APDFP such that we then have where $\Omega_{1},\Omega_{2}$ are constant related to diameter of $B_1,B_2$, respectively.

Figures (6)

  • Figure 1: The relative error of training and testing accuracy on different data sets.
  • Figure 1: The connection between PDFP and PGD variants. APDFP is the algorithm proposed in this work and it generalizes NAG when the matrix $B = I$. The dash arrow between IPDFP and FISTA means that IPDFP degenerate to FISTA when $B = I$ but lack theoretical generalization of FISTA.
  • Figure 2: The function value and PSNR versus the iteration number.
  • Figure 3: One slice of the reconstructed image produced by different algorithms. The corresponding PSNR values are indicated below each image.
  • Figure 4: The relative error of the objective function value and the testing accuracy with respect to the number of iterations for different values of $\lambda$ across various datasets.
  • ...and 1 more figures

Theorems & Definitions (16)

  • Remark 2.1
  • Remark 2.2
  • Definition 3.1
  • Theorem 3.2
  • Corollary 3.3
  • Remark 3.4
  • Remark 3.5
  • Remark 3.6
  • Lemma 6.1
  • Proof 1
  • ...and 6 more