Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Online Architecture Search for Compressed Sensing based on Hypergradient Descent

Ayano Nakai-Kasai, Yusuke Nakane, Tadashi Wadayama

TL;DR

This paper proposes HGD-AS-ISTA (Hypergradient Descent-AS-ISTA) and HGD-AS-FISTA (Hypergradient Descent-AS-ISTA) that use hypergradient descent, which is an online hyperparameter optimization method, to determine the structural parameters.

Abstract

AS-ISTA (Architecture Searched-Iterative Shrinkage Thresholding Algorithm) and AS-FISTA (AS-Fast ISTA) are compressed sensing algorithms introducing structural parameters to ISTA and FISTA to enable architecture search within the iterative process. The structural parameters are determined using deep unfolding, but this approach requires training data and the large overhead of training time. In this paper, we propose HGD-AS-ISTA (Hypergradient Descent-AS-ISTA) and HGD-AS-FISTA that use hypergradient descent, which is an online hyperparameter optimization method, to determine the structural parameters. Experimental results show that the proposed method improves performance of the conventional ISTA/FISTA while avoiding the need for re-training when the environment changes.

Online Architecture Search for Compressed Sensing based on Hypergradient Descent

TL;DR

This paper proposes HGD-AS-ISTA (Hypergradient Descent-AS-ISTA) and HGD-AS-FISTA (Hypergradient Descent-AS-ISTA) that use hypergradient descent, which is an online hyperparameter optimization method, to determine the structural parameters.

Abstract

AS-ISTA (Architecture Searched-Iterative Shrinkage Thresholding Algorithm) and AS-FISTA (AS-Fast ISTA) are compressed sensing algorithms introducing structural parameters to ISTA and FISTA to enable architecture search within the iterative process. The structural parameters are determined using deep unfolding, but this approach requires training data and the large overhead of training time. In this paper, we propose HGD-AS-ISTA (Hypergradient Descent-AS-ISTA) and HGD-AS-FISTA that use hypergradient descent, which is an online hyperparameter optimization method, to determine the structural parameters. Experimental results show that the proposed method improves performance of the conventional ISTA/FISTA while avoiding the need for re-training when the environment changes.
Paper Structure (13 sections, 33 equations, 2 figures, 1 table, 1 algorithm)

This paper contains 13 sections, 33 equations, 2 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Notation
  4. ISTA and FISTA
  5. Deep Unfolded ISTA and FISTA
  6. AS-ISTA and AS-FISTA
  7. Hypergradient Descent
  8. HGD-AS-ISTA and HGD-AS-FISTA
  9. Simulation Results
  10. Conclusion
  11. Hypergradient calculation for $\gamma^{(t)}$
  12. Hypergradient calculation for $w_r^{(t)}$ and $w_x^{(t)}$
  13. Hypergradient calculation for HGD-AS-FISTA

Figures (2)

  • Figure 1: Comparison of MSE vs iteration.
  • Figure 2: Examples of architecture selected for each algorithm.