Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DWA: Differential Wavelet Amplifier for Image Super-Resolution

Brian B. Moser, Stanislav Frolov, Federico Raue, Sebastian Palacio, Andreas Dengel

TL;DR

This work introduces Differential Wavelet Amplifier (DWA), a drop-in module for wavelet-based image Super-Resolution (SR), and shows its effectiveness by integrating it into existing SR models, e.g., DWSR and MWCNN, and demonstrates a clear improvement in classical SR tasks.

Abstract

This work introduces Differential Wavelet Amplifier (DWA), a drop-in module for wavelet-based image Super-Resolution (SR). DWA invigorates an approach recently receiving less attention, namely Discrete Wavelet Transformation (DWT). DWT enables an efficient image representation for SR and reduces the spatial area of its input by a factor of 4, the overall model size, and computation cost, framing it as an attractive approach for sustainable ML. Our proposed DWA model improves wavelet-based SR models by leveraging the difference between two convolutional filters to refine relevant feature extraction in the wavelet domain, emphasizing local contrasts and suppressing common noise in the input signals. We show its effectiveness by integrating it into existing SR models, e.g., DWSR and MWCNN, and demonstrate a clear improvement in classical SR tasks. Moreover, DWA enables a direct application of DWSR and MWCNN to input image space, reducing the DWT representation channel-wise since it omits traditional DWT.

DWA: Differential Wavelet Amplifier for Image Super-Resolution

TL;DR

This work introduces Differential Wavelet Amplifier (DWA), a drop-in module for wavelet-based image Super-Resolution (SR), and shows its effectiveness by integrating it into existing SR models, e.g., DWSR and MWCNN, and demonstrates a clear improvement in classical SR tasks.

Abstract

This work introduces Differential Wavelet Amplifier (DWA), a drop-in module for wavelet-based image Super-Resolution (SR). DWA invigorates an approach recently receiving less attention, namely Discrete Wavelet Transformation (DWT). DWT enables an efficient image representation for SR and reduces the spatial area of its input by a factor of 4, the overall model size, and computation cost, framing it as an attractive approach for sustainable ML. Our proposed DWA model improves wavelet-based SR models by leveraging the difference between two convolutional filters to refine relevant feature extraction in the wavelet domain, emphasizing local contrasts and suppressing common noise in the input signals. We show its effectiveness by integrating it into existing SR models, e.g., DWSR and MWCNN, and demonstrate a clear improvement in classical SR tasks. Moreover, DWA enables a direct application of DWSR and MWCNN to input image space, reducing the DWT representation channel-wise since it omits traditional DWT.
Paper Structure (14 sections, 4 equations, 4 figures, 2 tables)

This paper contains 14 sections, 4 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Discrete Wavelet Transform in SR
  4. Differential Amplifier
  5. Differential Convolutions
  6. Differential Wavelet Amplifier (DWA)
  7. Direct Application of DWA (DWA Direct)
  8. Experiments
  9. Experimental Setup
  10. Results
  11. Ablation Study
  12. Performance
  13. Visual Comparison
  14. Conclusion and Future Work

Figures (4)

  • Figure 1: Visualization of DWA. It takes the difference of two convolutional filters with a stride difference of at least 1, vertically and horizontally. Next, it concatenates the input with horizontal and vertical feature maps. In the end, it applies a final convolution.
  • Figure 2: Results of ablation study on BSDS100 with scaling factor 2x. We tested different configurations: Baseline, Direct (application on the image space), DWA, and DWA Direct (application on the image space).
  • Figure 3: Comparison of an HR ground truth image (BSDS100, 2x scaling), DWSR, and DWA. First row: the entire image space of the HR image and the corresponding reconstructions obtained by the SR models. Second row: zoomed-in regions within the images from the first row. Third row: residual image representing the difference between the LR and HR images. As a result, the DWA model captures edges and details closer to the ground truth residuals, as opposed to the DWSR model (also regarding color).
  • Figure 4: Feature maps of DWSR and DWA Direct. DWA Direct extracts local contrasts and variations more effectively, closer than DWSR to the residual target.