Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MTS-CSNet: Multiscale Tensor Factorization for Deep Compressive Sensing on RGB Images

Mehmet Yamac, Lei Xu, Serkan Kiranyaz, Moncef Gabbouj

TL;DR

MTS-CSNet addresses the inefficiency of block-wise sensing in compressive sensing for high-dimensional RGB images by introducing a learnable Multiscale Tensor Summation (MTS) operator. The framework performs end-to-end, non-iterative reconstruction in tensor space, using an MTS-based sensing layer, a nonlinear adjoint back-projection, and a lightweight MTSNet refinement. Key contributions include the MTS layer formulation with multiscale patch processing, a nonlinear adjoint back-projection via the MHG prior, and a compact refinement network that achieves state-of-the-art PSNR/SSIM with substantially lower computation than diffusion-based CS methods. Experiments on Urban100 and DIV2K demonstrate superior reconstruction quality and faster inference, validating the effectiveness and efficiency of multiscale tensor operators for high-dimensional CS. The work suggests that structured tensor factorizations can jointly model spatial and channel correlations across scales, enabling scalable CS for multidimensional data.

Abstract

Deep learning based compressive sensing (CS) methods typically learn sampling operators using convolutional or block wise fully connected layers, which limit receptive fields and scale poorly for high dimensional data. We propose MTSCSNet, a CS framework based on Multiscale Tensor Summation (MTS) factorization, a structured operator for efficient multidimensional signal processing. MTS performs mode-wise linear transformations with multiscale summation, enabling large receptive fields and effective modeling of cross-dimensional correlations. In MTSCSNet, MTS is first used as a learnable CS operator that performs linear dimensionality reduction in tensor space, with its adjoint defining the initial back-projection, and is then applied in the reconstruction stage to directly refine this estimate. This results in a simple feed-forward architecture without iterative or proximal optimization, while remaining parameter and computation efficient. Experiments on standard CS benchmarks show that MTSCSNet achieves state-of-the-art reconstruction performance on RGB images, with notable PSNR gains and faster inference, even compared to recent diffusion-based CS methods, while using a significantly more compact feed-forward architecture.

MTS-CSNet: Multiscale Tensor Factorization for Deep Compressive Sensing on RGB Images

TL;DR

MTS-CSNet addresses the inefficiency of block-wise sensing in compressive sensing for high-dimensional RGB images by introducing a learnable Multiscale Tensor Summation (MTS) operator. The framework performs end-to-end, non-iterative reconstruction in tensor space, using an MTS-based sensing layer, a nonlinear adjoint back-projection, and a lightweight MTSNet refinement. Key contributions include the MTS layer formulation with multiscale patch processing, a nonlinear adjoint back-projection via the MHG prior, and a compact refinement network that achieves state-of-the-art PSNR/SSIM with substantially lower computation than diffusion-based CS methods. Experiments on Urban100 and DIV2K demonstrate superior reconstruction quality and faster inference, validating the effectiveness and efficiency of multiscale tensor operators for high-dimensional CS. The work suggests that structured tensor factorizations can jointly model spatial and channel correlations across scales, enabling scalable CS for multidimensional data.

Abstract

Deep learning based compressive sensing (CS) methods typically learn sampling operators using convolutional or block wise fully connected layers, which limit receptive fields and scale poorly for high dimensional data. We propose MTSCSNet, a CS framework based on Multiscale Tensor Summation (MTS) factorization, a structured operator for efficient multidimensional signal processing. MTS performs mode-wise linear transformations with multiscale summation, enabling large receptive fields and effective modeling of cross-dimensional correlations. In MTSCSNet, MTS is first used as a learnable CS operator that performs linear dimensionality reduction in tensor space, with its adjoint defining the initial back-projection, and is then applied in the reconstruction stage to directly refine this estimate. This results in a simple feed-forward architecture without iterative or proximal optimization, while remaining parameter and computation efficient. Experiments on standard CS benchmarks show that MTSCSNet achieves state-of-the-art reconstruction performance on RGB images, with notable PSNR gains and faster inference, even compared to recent diffusion-based CS methods, while using a significantly more compact feed-forward architecture.
Paper Structure (13 sections, 6 equations, 4 figures, 4 tables)

This paper contains 13 sections, 6 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Preliminaries and Background
  3. Proposed Solution
  4. Multiscale Tensor Summation Layer
  5. MTS-Based Dimensional Reduction
  6. Adjoint MTS and Proxy Reconstruction
  7. MTSNet-Based Refinement Network
  8. Experiments
  9. Experimental Settings
  10. Quantitative and Visual Results
  11. Complexity Analysis
  12. Ablation Studies
  13. Conclusion

Figures (4)

  • Figure 1: Overall architecture of the proposed MTS-CSNet. An MTS-layer-based autoencoder performs compressive sensing and adjoint back-projection, followed by a lightweight MTSNet refinement network for feed-forward reconstruction.
  • Figure 2: Visual comparison of CS reconstruction results on the "img034" image from Urban100 at sampling rate 0.3.
  • Figure 3: Comparison of CS recovery results among various competing methods on "img096" image from Urban100 at sampling rate 0.3.
  • Figure 4: Comparison between proxy reconstructions (adjoint output) and final results produced by MTS-CSNet.