Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dynamic Thresholding Algorithm with Memory for Linear Inverse Problems

Zhong-Feng Sun, Yun-Bin Zhao, Jin-Chuan Zhou, Zheng-Hai Huang

TL;DR

It turns out that DTAM can locate the solution of linear inverse problems if the matrix involved satisfies the restricted isometry property, and it works faster than the ROTP-type algorithms especially when the sparsity level of signal is relatively low.

Abstract

The relaxed optimal $k$-thresholding pursuit (ROTP) is a recent algorithm for linear inverse problems. This algorithm is based on the optimal $k$-thresholding technique which performs vector thresholding and error metric reduction simultaneously. Although ROTP can be used to solve small to medium-sized linear inverse problems, the computational cost of this algorithm is high when solving large-scale problems. By merging the optimal $k$-thresholding technique and iterative method with memory as well as optimization with sparse search directions, we propose the so-called dynamic thresholding algorithm with memory (DTAM), which iteratively and dynamically selects vector bases to construct the problem solution. At every step, the algorithm uses more than one or all iterates generated so far to construct a new search direction, and solves only the small-sized quadratic subproblems at every iteration. Thus the computational complexity of DTAM is remarkably lower than that of ROTP-type methods. It turns out that DTAM can locate the solution of linear inverse problems if the matrix involved satisfies the restricted isometry property. Experiments on synthetic data, audio signal reconstruction and image denoising demonstrate that the proposed algorithm performs comparably to several mainstream thresholding and greedy algorithms, and it works much faster than the ROTP-type algorithms especially when the sparsity level of signal is relatively low.

Dynamic Thresholding Algorithm with Memory for Linear Inverse Problems

TL;DR

It turns out that DTAM can locate the solution of linear inverse problems if the matrix involved satisfies the restricted isometry property, and it works faster than the ROTP-type algorithms especially when the sparsity level of signal is relatively low.

Abstract

The relaxed optimal -thresholding pursuit (ROTP) is a recent algorithm for linear inverse problems. This algorithm is based on the optimal -thresholding technique which performs vector thresholding and error metric reduction simultaneously. Although ROTP can be used to solve small to medium-sized linear inverse problems, the computational cost of this algorithm is high when solving large-scale problems. By merging the optimal -thresholding technique and iterative method with memory as well as optimization with sparse search directions, we propose the so-called dynamic thresholding algorithm with memory (DTAM), which iteratively and dynamically selects vector bases to construct the problem solution. At every step, the algorithm uses more than one or all iterates generated so far to construct a new search direction, and solves only the small-sized quadratic subproblems at every iteration. Thus the computational complexity of DTAM is remarkably lower than that of ROTP-type methods. It turns out that DTAM can locate the solution of linear inverse problems if the matrix involved satisfies the restricted isometry property. Experiments on synthetic data, audio signal reconstruction and image denoising demonstrate that the proposed algorithm performs comparably to several mainstream thresholding and greedy algorithms, and it works much faster than the ROTP-type algorithms especially when the sparsity level of signal is relatively low.

Paper Structure

This paper contains 11 sections, 9 theorems, 101 equations, 3 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminary and Algorithms
  3. Basic inequalities
  4. Generalized mean function
  5. Algorithms
  6. Error bound of DTAM
  7. Numerical experiments
  8. Experiments with synthetic data
  9. Reconstruction of audio signal
  10. Image denoising
  11. Conclusions

Key Result

Lemma 2.2

F11Z20 Let $u \in \mathbb{R}^n$ and $v \in \mathbb{R}^m$ be two vectors, $s \in N$ be a positive integer and $W\subseteq N$ be an index set.

Figures (3)

  • Figure 1: Comparison of success frequencies and runtime on synthetic data, and $T$ is the average CPU time (in seconds) for recovery.
  • Figure 2: Reconstruction of an audio signal by DTAM with $\kappa=0.5;$ The first row is the original signal, and the second one demonstrates both the original signal ( blue) and the reconstructed one by DTAM ( red).
  • Figure 3: Performance of DTAM on image denoising.

Theorems & Definitions (18)

  • Definition 2.1
  • Lemma 2.2
  • Lemma 2.3
  • Lemma 2.4
  • Lemma 2.5
  • Example 2.6
  • Remark 2.7
  • Remark 2.8
  • Lemma 3.1
  • Lemma 3.2
  • ...and 8 more