Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust Non-negative Proximal Gradient Algorithm for Inverse Problems

Hanzhang Wang, Zonglin Liu, Jingyi Xu, Chenyang Wang, Zhiwei Zhong, Qiangqiang Shen

TL;DR

This work proposes a novel multiplicative update proximal gradient algorithm (SSO-PGA) with convergence guarantees, designed for robustness in non-negative inverse problems and significantly surpasses traditional PGA and other state-of-the-art algorithms, ensuring superior performance and stability.

Abstract

Proximal gradient algorithms (PGA), while foundational for inverse problems like image reconstruction, often yield unstable convergence and suboptimal solutions by violating the critical non-negativity constraint. We identify the gradient descent step as the root cause of this issue, which introduces negative values and induces high sensitivity to hyperparameters. To overcome these limitations, we propose a novel multiplicative update proximal gradient algorithm (SSO-PGA) with convergence guarantees, which is designed for robustness in non-negative inverse problems. Our key innovation lies in superseding the gradient descent step with a learnable sigmoid-based operator, which inherently enforces non-negativity and boundedness by transforming traditional subtractive updates into multiplicative ones. This design, augmented by a sliding parameter for enhanced stability and convergence, not only improves robustness but also boosts expressive capacity and noise immunity. We further formulate a degradation model for multi-modal restoration and derive its SSO-PGA-based optimization algorithm, which is then unfolded into a deep network to marry the interpretability of optimization with the power of deep learning. Extensive numerical and real-world experiments demonstrate that our method significantly surpasses traditional PGA and other state-of-the-art algorithms, ensuring superior performance and stability.

Robust Non-negative Proximal Gradient Algorithm for Inverse Problems

TL;DR

This work proposes a novel multiplicative update proximal gradient algorithm (SSO-PGA) with convergence guarantees, designed for robustness in non-negative inverse problems and significantly surpasses traditional PGA and other state-of-the-art algorithms, ensuring superior performance and stability.

Abstract

Proximal gradient algorithms (PGA), while foundational for inverse problems like image reconstruction, often yield unstable convergence and suboptimal solutions by violating the critical non-negativity constraint. We identify the gradient descent step as the root cause of this issue, which introduces negative values and induces high sensitivity to hyperparameters. To overcome these limitations, we propose a novel multiplicative update proximal gradient algorithm (SSO-PGA) with convergence guarantees, which is designed for robustness in non-negative inverse problems. Our key innovation lies in superseding the gradient descent step with a learnable sigmoid-based operator, which inherently enforces non-negativity and boundedness by transforming traditional subtractive updates into multiplicative ones. This design, augmented by a sliding parameter for enhanced stability and convergence, not only improves robustness but also boosts expressive capacity and noise immunity. We further formulate a degradation model for multi-modal restoration and derive its SSO-PGA-based optimization algorithm, which is then unfolded into a deep network to marry the interpretability of optimization with the power of deep learning. Extensive numerical and real-world experiments demonstrate that our method significantly surpasses traditional PGA and other state-of-the-art algorithms, ensuring superior performance and stability.

Paper Structure

This paper contains 26 sections, 5 theorems, 43 equations, 27 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. SSO-enhanced Proximal Gradient Algorithm
  5. Formulation and Optimization
  6. Deep Unfolding Network
  7. Experiments
  8. Comparison with Traditional Proximal Gradient Algorithm
  9. Numerical Experiments
  10. Real-world Vision Experiments
  11. Comparison with SOTAs
  12. Multispectral Image Fusion
  13. Flash Guided Non-Flash Image Denoising
  14. Ablation Study
  15. Conclusion
  16. ...and 11 more sections

Key Result

Theorem 1

There exists $\rho_i > 0$ such that the SSO update rule is equivalent to a standard gradient descent step:

Figures (27)

  • Figure 1: Overview of our method. (a) A schematic comparison between PGA and SSO-PGA in the gradient descent process. Compared to PGA, SSO-PGA benefits from the non-negativity constraint, yielding more stable solutions and demonstrating a faster convergence trajectory. (b) Comparison of test L1 loss curves between PGA and SSO-PGA on the WV3 dataset in the image fusion task over training epochs, with a zoomed-in view highlighting the reconstructed results at epoch 200. SSO-PGA exhibits a more stable training process and achieves superior fusion quality.
  • Figure 2: SSO working mechanism and its function curves under different $\alpha$ values.
  • Figure 3: The network architecture of our method. (a) SSO-PGA consists of $T$ iterative steps, where each iteration includes (b) the update of $\boldsymbol{\mathcal{H}}$ and (c) the update of $\boldsymbol{\mathcal{T}}$. (d) The detailed network architecture of SSO-PGA, including the init module, MSFB block, and $Prox_{\phi}(\cdot)$ from left to right.
  • Figure 4: Comparison of numerical simulation results for SSO-PGA and PGA on Problem I.
  • Figure 5: Comparison of numerical simulation results for SSO-PGA and PGA on Problem II.
  • ...and 22 more figures

Theorems & Definitions (12)

  • Definition 1
  • Definition 2
  • Theorem 1
  • Lemma 1
  • Lemma 2
  • Lemma 3
  • Theorem 2
  • proof
  • proof
  • proof
  • ...and 2 more