Training-free Ultra Small Model for Universal Sparse Reconstruction in Compressed Sensing

TL;DR

This paper proposes ultra-small artificial neural models called coefficients learning (CL), enabling training-free and rapid sparse reconstruction while perfectly inheriting the generality and interpretability of traditional iterative methods, bringing new feature of incorporating prior knowledges.

Abstract

Pre-trained large models attract widespread attention in recent years, but they face challenges in applications that require high interpretability or have limited resources, such as physical sensing, medical imaging, and bioinformatics. Compressed Sensing (CS) is a well-proved theory that drives many recent breakthroughs in these applications. However, as a typical under-determined linear system, CS suffers from excessively long sparse reconstruction times when using traditional iterative methods, particularly with large-scale data. Current AI methods like deep unfolding fail to substitute them because pre-trained models exhibit poor generality beyond their training conditions and dataset distributions, or lack interpretability. Instead of following the big model fervor, this paper proposes ultra-small artificial neural models called coefficients learning (CL), enabling training-free and rapid sparse reconstruction while perfectly inheriting the generality and interpretability of traditional iterative methods, bringing new feature of incorporating prior knowledges. In CL, a signal of length $n$ only needs a minimal of $n$ trainable parameters. A case study model called CLOMP is implemented for evaluation. Experiments are conducted on both synthetic and real one-dimensional and two-dimensional signals, demonstrating significant improvements in efficiency and accuracy. Compared to representative iterative methods, CLOMP improves efficiency by 100 to 1000 folds for large-scale data. Test results on eight diverse image datasets indicate that CLOMP improves structural similarity index by 292%, 98%, 45% for sampling rates of 0.1, 0.3, 0.5, respectively. We believe this method can truly usher CS reconstruction into the AI era, benefiting countless under-determined linear systems that rely on sparse solution.

Figures (10)

• Figure 1: Reconstruction accuracy based on SSIM in relation to the sparse rate ($Kr$), sampling rate ($Sr$) and signal length ($n$) for 1D signals. ($\bf{a}$) is an example of a synthetic 1D signal with $n=10^3$ and $Kr=0.2$, where the coefficients $\bf{s}$ and the time-domain signal $\bf{x}$ are presented. The SSIM values for a demo method of the proposed framework (CLOMP) and its mother method OMP, and IHT are presented in ($\bf{b-c}$). ($\bf{c}$) shows the typical best, middle, and worst cases with the combination of $Kr$ and $Sr$ from top to bottom. ($\bf{d,e}$) shows the SSIM ratio of CLOMP to OMP for $n=10^4$ and $n=10^3$, respectively.
• Figure 2: The reconstruction time of the proposed CLOMP and other methods versus $Sr$, $Kr$, $n$ and signal quantity for 1D signals. ($\bf{a})$ is a 3D plot for time versus $Sr$ and $Kr$ under different $n$. Time versus $n$ for typical best, middle, and worst cases are denoted with 'o', '$\times$', and line in ($\bf{b}$), respectively. ($\bf{d,e}$) show the time versus signal quantity under worst and best cases for the combination of $Sr$ and $Kr$, respectively. The implementation of OMP and IHT under $n=10^4$ on an Intel Xeon Platinum 8352Y CPU is also highlighted for the case where the signal quantity is 1. ($\bf{c,f}$) present the time ratio of CLOMP to OMP for $n=10^4$ and $n=10^3$, respectively.
• Figure 3: Examples of reconstructions with a length of 100 on real 1D datasets.The ground truths are denoted as GT in ($\bf{a-i}$). The sparse rates ($Kr$) of the full signal length ($n$) are given in the figure. SSIM and PCC of the demo signal are calculated and presented in the square brackets. ($\bf{j}$) provides further evaluation for the proposed CLOMP and its mother method OMP, where the minimum time and $Sr$ to reach SSIM>0.9 are tested for the demo signal in ($\bf{a-i}$), where $T_h$ and maximum iteration number are freely set for each method.
• Figure 4: The reconstruction accuracy based on SSIM in relation to image resolution, $Sr$ and $Kr$ for 2D signals. ($\bf{a-c}$) are for the synthetic images with $Kr$=0.55, 0.69, 0.83, respectively. ($\bf{d-f}$) are the corresponding SSIM ratios of CLOMP to OMP or $Kr$=0.55, 0.69, 0.83, respectively.
• Figure 5: Reconstruction time versus image resolution, $Sr$ and $Kr$ for 2D signals. ($\bf{a-c}$) are for low, middle, and high sampling rates with $Sr$=0.1, 0.3, 0.5, respectively.