Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Image inpainting for corrupted images by using the semi-super resolution GAN

Mehrshad Momen-Tayefeh, Mehrdad Momen-Tayefeh, Amir Ali Ghafourian Ghahramani

TL;DR

The paper tackles image inpainting for corrupted images by introducing SSRGAN, a lighter variant of SRGAN designed to reconstruct missing pixels through adversarial learning. SSRGAN is trained with randomly corrupted pixels to learn robust pixel-level reconstruction, using a generator with residual blocks and pixel-shuffle upscaling and a discriminator with a multi-block architecture. The model is evaluated on Oxford IIIT Pets, Caltech101, and Flower102, reporting NMSE and PSNR across corruption levels and showing increasing reconstruction quality with higher corruption levels handled. The work demonstrates a practical, parameter-efficient approach for inpainting under varying damage and proposes NMSE as a normalized evaluation metric alongside PSNR.

Abstract

Image inpainting is a valuable technique for enhancing images that have been corrupted. The primary challenge in this research revolves around the extent of corruption in the input image that the deep learning model must restore. To address this challenge, we introduce a Generative Adversarial Network (GAN) for learning and replicating the missing pixels. Additionally, we have developed a distinct variant of the Super-Resolution GAN (SRGAN), which we refer to as the Semi-SRGAN (SSRGAN). Furthermore, we leveraged three diverse datasets to assess the robustness and accuracy of our proposed model. Our training process involves varying levels of pixel corruption to attain optimal accuracy and generate high-quality images.

Image inpainting for corrupted images by using the semi-super resolution GAN

TL;DR

The paper tackles image inpainting for corrupted images by introducing SSRGAN, a lighter variant of SRGAN designed to reconstruct missing pixels through adversarial learning. SSRGAN is trained with randomly corrupted pixels to learn robust pixel-level reconstruction, using a generator with residual blocks and pixel-shuffle upscaling and a discriminator with a multi-block architecture. The model is evaluated on Oxford IIIT Pets, Caltech101, and Flower102, reporting NMSE and PSNR across corruption levels and showing increasing reconstruction quality with higher corruption levels handled. The work demonstrates a practical, parameter-efficient approach for inpainting under varying damage and proposes NMSE as a normalized evaluation metric alongside PSNR.

Abstract

Image inpainting is a valuable technique for enhancing images that have been corrupted. The primary challenge in this research revolves around the extent of corruption in the input image that the deep learning model must restore. To address this challenge, we introduce a Generative Adversarial Network (GAN) for learning and replicating the missing pixels. Additionally, we have developed a distinct variant of the Super-Resolution GAN (SRGAN), which we refer to as the Semi-SRGAN (SSRGAN). Furthermore, we leveraged three diverse datasets to assess the robustness and accuracy of our proposed model. Our training process involves varying levels of pixel corruption to attain optimal accuracy and generate high-quality images.
Paper Structure (8 sections, 7 equations, 6 figures, 2 tables)

This paper contains 8 sections, 7 equations, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Approach
  3. Methods
  4. Network architecture
  5. Loss Function
  6. Dataset
  7. Results
  8. Conclusion

Figures (6)

  • Figure 1: An instance of reconstructing the missed pixels by using our method.
  • Figure 2: The proposed Semi-Super Resolution GAN (SSRGAN) model for image inpainting. (a) The SSRGAN generator. (b) The discriminator for training the SSRGAN generator.
  • Figure 3: A representation of levels pixel corruption from $30\%$ to $80\%$
  • Figure 4: The result of output images from Oxfordiiitpet, Caltec101, and Flower102 datasets on our SSRGAN for each degree of eliminated pixels.
  • Figure 5: Changes of NMSE by increasing the level of pixels off for three datasets.
  • ...and 1 more figures