Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beware of Aliases -- Signal Preservation is Crucial for Robust Image Restoration

Shashank Agnihotri, Julia Grabinski, Janis Keuper, Margret Keuper

TL;DR

This work tackles the vulnerability of image restoration models to aliasing artifacts by enforcing alias-free information paths through frequency-domain downsampling and upsampling. It introduces FrequencyPreservedPooling and FreqAvgUp, collectively BOA-Pooling, and enforces encoder–decoder symmetry to preserve high-frequency content while stabilizing low-frequency projections. The BOA-Restormer shows improved robustness against white-box adversarial attacks (PGD and CosPGD) on the GoPro image deblurring task, with reduced spectral artifacts and competitive restoration quality versus state-of-the-art baselines. The approach highlights the importance of signal-processing principles in pixel-wise vision tasks and suggests that carefully balancing low- and high-frequency information can yield robust restorations suitable for safety-critical applications.

Abstract

Image restoration networks are usually comprised of an encoder and a decoder, responsible for aggregating image content from noisy, distorted data and to restore clean, undistorted images, respectively. Data aggregation as well as high-resolution image generation both usually come at the risk of involving aliases, i.e.~standard architectures put their ability to reconstruct the model input in jeopardy to reach high PSNR values on validation data. The price to be paid is low model robustness. In this work, we show that simply providing alias-free paths in state-of-the-art reconstruction transformers supports improved model robustness at low costs on the restoration performance. We do so by proposing BOA-Restormer, a transformer-based image restoration model that executes downsampling and upsampling operations partly in the frequency domain to ensure alias-free paths along the entire model while potentially preserving all relevant high-frequency information.

Beware of Aliases -- Signal Preservation is Crucial for Robust Image Restoration

TL;DR

This work tackles the vulnerability of image restoration models to aliasing artifacts by enforcing alias-free information paths through frequency-domain downsampling and upsampling. It introduces FrequencyPreservedPooling and FreqAvgUp, collectively BOA-Pooling, and enforces encoder–decoder symmetry to preserve high-frequency content while stabilizing low-frequency projections. The BOA-Restormer shows improved robustness against white-box adversarial attacks (PGD and CosPGD) on the GoPro image deblurring task, with reduced spectral artifacts and competitive restoration quality versus state-of-the-art baselines. The approach highlights the importance of signal-processing principles in pixel-wise vision tasks and suggests that carefully balancing low- and high-frequency information can yield robust restorations suitable for safety-critical applications.

Abstract

Image restoration networks are usually comprised of an encoder and a decoder, responsible for aggregating image content from noisy, distorted data and to restore clean, undistorted images, respectively. Data aggregation as well as high-resolution image generation both usually come at the risk of involving aliases, i.e.~standard architectures put their ability to reconstruct the model input in jeopardy to reach high PSNR values on validation data. The price to be paid is low model robustness. In this work, we show that simply providing alias-free paths in state-of-the-art reconstruction transformers supports improved model robustness at low costs on the restoration performance. We do so by proposing BOA-Restormer, a transformer-based image restoration model that executes downsampling and upsampling operations partly in the frequency domain to ensure alias-free paths along the entire model while potentially preserving all relevant high-frequency information.
Paper Structure (26 sections, 18 equations, 10 figures, 5 tables)

This paper contains 26 sections, 18 equations, 10 figures, 5 tables.

Table of Contents

  1. Related Work
  2. Adversarial Robustness
  3. Aliasing Artifacts
  4. Sampling Methods
  5. Frequency Learning in Image Restoration
  6. Method
  7. Downsampling
  8. Upsampling
  9. Experiments
  10. Experimental Setup
  11. Downstream Task.
  12. Dataset.
  13. Evaluation Metrics.
  14. Networks.
  15. Adversarial Attacks.
  16. ...and 11 more sections

Figures (10)

  • Figure 1: Comparing images deblurred by image restoration models after 10 iterations of PGD attack. We observe strong spectral artifacts in the restored images are fixed by our proposed "FrequencyPreservedPooling" (downsampling) and "FreqAvgUp" (upsampling) operations. The PSNR values reported are averaged over the entire test set.
  • Figure 2: A visual representation of the proposed sampling operations. Left, we show the flow diagram for our proposed downsampling operation "FrequencyPreservedPooling" used in the encoder of the image restoration model's architecture. Right, we show the flow diagram for our proposed upsampling operation "FreqAvgUp" used in the decoder of the BOA architecture.
  • Figure 3: BOA-network architecture using our up- and downsampling from \ref{['boa']} and skip connections between encoder and decoder.
  • Figure 4: Comparing the proposed architectural design choices qualitatively against other baselines for image restoration, on normal blurry input images and input images adversarially attacked using CosPGD. Symbolic notations are same as those in \ref{['tab:restormer_variants']}.
  • Figure 5: Different downsampling methods being compared qualitatively on normal blurry input images and input images adversarially attacked using CosPGD for five iterations. Symbolic notations are same as those in \ref{['tab:restormer_encoder_ablation']}.
  • ...and 5 more figures