Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Unrolling Networks with Recurrent Momentum Acceleration for Nonlinear Inverse Problems

Qingping Zhou, Jiayu Qian, Junqi Tang, Jinglai Li

TL;DR

This work tackles nonlinear inverse problems where traditional deep unrolling networks (DuNets) struggle due to gradient nonlinearity. It introduces recurrent momentum acceleration (RMA) by embedding a learned LSTM-RNN to compute a velocity term that aggregates gradient history, enabling LPGD-RMA and LPD-RMA. Across nonlinear deconvolution and electrical impedance tomography (EIT), RMA consistently improves reconstruction quality, with LPD-RMA often delivering the best performance and showing data-efficiency advantages over standard MA variants. The proposed approach provides a scalable, data-driven regularization mechanism that enhances stability and accuracy in strongly ill-posed, nonlinear inverse problems, with potential applicability to other unrolled frameworks.

Abstract

Combining the strengths of model-based iterative algorithms and data-driven deep learning solutions, deep unrolling networks (DuNets) have become a popular tool to solve inverse imaging problems. While DuNets have been successfully applied to many linear inverse problems, nonlinear problems tend to impair the performance of the method. Inspired by momentum acceleration techniques that are often used in optimization algorithms, we propose a recurrent momentum acceleration (RMA) framework that uses a long short-term memory recurrent neural network (LSTM-RNN) to simulate the momentum acceleration process. The RMA module leverages the ability of the LSTM-RNN to learn and retain knowledge from the previous gradients. We apply RMA to two popular DuNets -- the learned proximal gradient descent (LPGD) and the learned primal-dual (LPD) methods, resulting in LPGD-RMA and LPD-RMA respectively. We provide experimental results on two nonlinear inverse problems: a nonlinear deconvolution problem, and an electrical impedance tomography problem with limited boundary measurements. In the first experiment we have observed that the improvement due to RMA largely increases with respect to the nonlinearity of the problem. The results of the second example further demonstrate that the RMA schemes can significantly improve the performance of DuNets in strongly ill-posed problems.

Deep Unrolling Networks with Recurrent Momentum Acceleration for Nonlinear Inverse Problems

TL;DR

This work tackles nonlinear inverse problems where traditional deep unrolling networks (DuNets) struggle due to gradient nonlinearity. It introduces recurrent momentum acceleration (RMA) by embedding a learned LSTM-RNN to compute a velocity term that aggregates gradient history, enabling LPGD-RMA and LPD-RMA. Across nonlinear deconvolution and electrical impedance tomography (EIT), RMA consistently improves reconstruction quality, with LPD-RMA often delivering the best performance and showing data-efficiency advantages over standard MA variants. The proposed approach provides a scalable, data-driven regularization mechanism that enhances stability and accuracy in strongly ill-posed, nonlinear inverse problems, with potential applicability to other unrolled frameworks.

Abstract

Combining the strengths of model-based iterative algorithms and data-driven deep learning solutions, deep unrolling networks (DuNets) have become a popular tool to solve inverse imaging problems. While DuNets have been successfully applied to many linear inverse problems, nonlinear problems tend to impair the performance of the method. Inspired by momentum acceleration techniques that are often used in optimization algorithms, we propose a recurrent momentum acceleration (RMA) framework that uses a long short-term memory recurrent neural network (LSTM-RNN) to simulate the momentum acceleration process. The RMA module leverages the ability of the LSTM-RNN to learn and retain knowledge from the previous gradients. We apply RMA to two popular DuNets -- the learned proximal gradient descent (LPGD) and the learned primal-dual (LPD) methods, resulting in LPGD-RMA and LPD-RMA respectively. We provide experimental results on two nonlinear inverse problems: a nonlinear deconvolution problem, and an electrical impedance tomography problem with limited boundary measurements. In the first experiment we have observed that the improvement due to RMA largely increases with respect to the nonlinearity of the problem. The results of the second example further demonstrate that the RMA schemes can significantly improve the performance of DuNets in strongly ill-posed problems.
Paper Structure (30 sections, 20 equations, 9 figures, 4 tables, 6 algorithms)

This paper contains 30 sections, 20 equations, 9 figures, 4 tables, 6 algorithms.

Table of Contents

  1. Introduction
  2. Deep unrolling networks
  3. Learned proximal gradient descent method
  4. Learned primal-dual method
  5. Deep unrolling networks with momentum acceleration
  6. Momentum acceleration methods
  7. Explicit momentum acceleration
  8. Momentum acceleration via RNN
  9. LPGD and LPD with MA
  10. LPGD
  11. LPD
  12. Experiments and Results
  13. Implementation details
  14. A nonlinear deconvolution problem
  15. Problem setting
  16. ...and 15 more sections

Figures (9)

  • Figure 1: The model architectures of DuNets-RMA. The RMA module is constructed via a deep LSTM-RNN, denoted as $\Xi_{\vartheta}$.
  • Figure 2: Deconvolution results and their corresponding MSE values for all DuNets-RMA methods.
  • Figure 3: The MSE results plotted against the data size for $a=1$.
  • Figure 4: EIT reconstruction results of four testing samples in Case 1 (2 inclusions). From left to right: ground truth, GN, PDIPM-TV, LPGD-RMA, LPGDSW-RMA, and LPD-RMA.
  • Figure 5: EIT reconstruction results of four testing samples in Case 2 (4 inclusions). From left to right: ground truth, GN, PDIPM-TV, LPGD-RMA, LPGDSW-RMA, and LPD-RMA.
  • ...and 4 more figures