Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

One-Shot Image Restoration

Deborah Pereg

TL;DR

The paper tackles the data efficiency and domain shift challenges in supervised image restoration by proposing a one-shot patch-based learning framework trained from a single degraded input–ground-truth pair. It combines an RNN based encoder–decoder with sparse coding insights to realize a learned proximal update, and supports Patch2Pixel and Patch2Patch workflows along with UNet and GAN enhancements. Experiments on image deblurring and single image super-resolution show competitive PSNR/SSIM against established baselines while dramatically reducing training time and memory requirements, enabling near real-time adaptation. The work suggests broad applicability to other signals and modalities and provides a theoretical and empirical basis for internal recurrence and typical-set representations to support generalization.

Abstract

Image restoration, or inverse problems in image processing, has long been an extensively studied topic. In recent years supervised learning approaches have become a popular strategy attempting to tackle this task. Unfortunately, most supervised learning-based methods are highly demanding in terms of computational resources and training data (sample complexity). In addition, trained models are sensitive to domain changes, such as varying acquisition systems, signal sampling rates, resolution and contrast. In this work, we try to answer a fundamental question: Can supervised learning models generalize well solely by learning from one image or even part of an image? If so, then what is the minimal amount of patches required to achieve acceptable generalization? To this end, we focus on an efficient patch-based learning framework that requires a single image input-output pair for training. Experimental results demonstrate the applicability, robustness and computational efficiency of the proposed approach for supervised image deblurring and super-resolution. Our results showcase significant improvement of learning models' sample efficiency, generalization and time complexity, that can hopefully be leveraged for future real-time applications, and applied to other signals and modalities.

One-Shot Image Restoration

TL;DR

The paper tackles the data efficiency and domain shift challenges in supervised image restoration by proposing a one-shot patch-based learning framework trained from a single degraded input–ground-truth pair. It combines an RNN based encoder–decoder with sparse coding insights to realize a learned proximal update, and supports Patch2Pixel and Patch2Patch workflows along with UNet and GAN enhancements. Experiments on image deblurring and single image super-resolution show competitive PSNR/SSIM against established baselines while dramatically reducing training time and memory requirements, enabling near real-time adaptation. The work suggests broad applicability to other signals and modalities and provides a theoretical and empirical basis for internal recurrence and typical-set representations to support generalization.

Abstract

Image restoration, or inverse problems in image processing, has long been an extensively studied topic. In recent years supervised learning approaches have become a popular strategy attempting to tackle this task. Unfortunately, most supervised learning-based methods are highly demanding in terms of computational resources and training data (sample complexity). In addition, trained models are sensitive to domain changes, such as varying acquisition systems, signal sampling rates, resolution and contrast. In this work, we try to answer a fundamental question: Can supervised learning models generalize well solely by learning from one image or even part of an image? If so, then what is the minimal amount of patches required to achieve acceptable generalization? To this end, we focus on an efficient patch-based learning framework that requires a single image input-output pair for training. Experimental results demonstrate the applicability, robustness and computational efficiency of the proposed approach for supervised image deblurring and super-resolution. Our results showcase significant improvement of learning models' sample efficiency, generalization and time complexity, that can hopefully be leveraged for future real-time applications, and applied to other signals and modalities.
Paper Structure (11 sections, 21 equations, 11 figures, 3 tables)

This paper contains 11 sections, 21 equations, 11 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. RNN Framework
  4. Sparse Coding $\&$ Iterative Shrinkage Algorithms
  5. RNN Analyzed via Sparse Coding
  6. Patch-based Learning
  7. Numerical Experiments
  8. Runtime
  9. System Mismatch
  10. Conclusions
  11. Sparse Representations

Figures (11)

  • Figure 1: Illustration of the proposed patch-to-patch RNN encoder-decoder.
  • Figure 2: Visual comparison of deblurring of the image starfish: (a) Ground truth; (b) input, 24.8dB; (c) RED-SD, 32.42dB; (d) RNN, 29.18dB; (e) RNN-GAN, 29.11dB; (f) UNET-GAN, 30.94 dB.
  • Figure 3: Visual comparison of deblurring of the parrot image: (a) Ground truth; (b) input, 25.33dB; (c) RED-SD, 33.18dB; (d) RNN, 30.98dB; (e) RNN-GAN, 30.30dB; (f) UNET-GAN, 32.22dB.
  • Figure 4: Visual comparison of deblurring of the image starfish for different noise levels: (a) Ground truth; (b) input, 24.25dB, $\sigma_n=4.24$; (c) RNN, 28.07dB ; (d) input, 23.02dB, $\sigma_n=9.90$; (e) RNN-GAN, 26.36dB .
  • Figure 5: Visual comparison of super resolution of the image butterfly: (a) Ground truth; (b) Bicubic interpolation, 19.44dB; (c) RED-SD (DnCNN), 23.57dB; (d) RNN, 22.84dB; (e) RNN-GAN, 22.45dB; (f) UNET-GAN, 22.83dB.
  • ...and 6 more figures