Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quaternion Generative Adversarial Neural Networks and Applications to Color Image Inpainting

Duan Wang, Dandan Zhu, Meixiang Zhao, Zhigang Jia

TL;DR

The paper tackles color image inpainting with large missing regions by preserving inter-channel color correlations through a Quaternion Generative Adversarial Network (QGAN). It introduces quaternion deconvolution and quaternion batch normalization to build a quaternion-valued generator and proposes quaternion context and prior losses to guide realistic inpainting, with training stability demonstrated over standard GANs. Empirical results on SVHN and CelebA show that QGAN achieves higher PSNR and SSIM in challenging central- and diagonal-block missing scenarios and exhibits robust, stable training. Overall, the work offers a principled quaternion-based framework that improves color-consistent inpainting for large-area missing data, outperforming prior quaternion-based and real-valued approaches.

Abstract

Color image inpainting is a challenging task in imaging science. The existing method is based on real operation, and the red, green and blue channels of the color image are processed separately, ignoring the correlation between each channel. In order to make full use of the correlation between each channel, this paper proposes a Quaternion Generative Adversarial Neural Network (QGAN) model and related theory, and applies it to solve the problem of color image inpainting with large area missing. Firstly, the definition of quaternion deconvolution is given and the quaternion batch normalization is proposed. Secondly, the above two innovative modules are applied to generate adversarial networks to improve stability. Finally, QGAN is applied to color image inpainting and compared with other state-of-the-art algorithms. The experimental results show that QGAN has superiority in color image inpainting with large area missing.

Quaternion Generative Adversarial Neural Networks and Applications to Color Image Inpainting

TL;DR

The paper tackles color image inpainting with large missing regions by preserving inter-channel color correlations through a Quaternion Generative Adversarial Network (QGAN). It introduces quaternion deconvolution and quaternion batch normalization to build a quaternion-valued generator and proposes quaternion context and prior losses to guide realistic inpainting, with training stability demonstrated over standard GANs. Empirical results on SVHN and CelebA show that QGAN achieves higher PSNR and SSIM in challenging central- and diagonal-block missing scenarios and exhibits robust, stable training. Overall, the work offers a principled quaternion-based framework that improves color-consistent inpainting for large-area missing data, outperforming prior quaternion-based and real-valued approaches.

Abstract

Color image inpainting is a challenging task in imaging science. The existing method is based on real operation, and the red, green and blue channels of the color image are processed separately, ignoring the correlation between each channel. In order to make full use of the correlation between each channel, this paper proposes a Quaternion Generative Adversarial Neural Network (QGAN) model and related theory, and applies it to solve the problem of color image inpainting with large area missing. Firstly, the definition of quaternion deconvolution is given and the quaternion batch normalization is proposed. Secondly, the above two innovative modules are applied to generate adversarial networks to improve stability. Finally, QGAN is applied to color image inpainting and compared with other state-of-the-art algorithms. The experimental results show that QGAN has superiority in color image inpainting with large area missing.
Paper Structure (16 sections, 2 theorems, 41 equations, 6 figures, 3 tables, 3 algorithms)

This paper contains 16 sections, 2 theorems, 41 equations, 6 figures, 3 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Quaternion Generative Adversarial Network
  4. Quaternion deconvolution
  5. Quaternion batch normalization
  6. Quaternion Generative Adversarial Network
  7. The QGAN Method for Color Image inpainting
  8. Quaternion context loss function
  9. Quaternion priori loss function
  10. Image Inpainting Model
  11. Experiments results
  12. Pre-training
  13. QGAN Application to Color Image inpainting with Missing Central Block Pixels
  14. QGAN Application to color image inpainting with missing diag block pixels
  15. Stability testing of more inpainting images
  16. ...and 1 more sections

Key Result

Theorem 2.1

s1 Let ${\bf q}$ be a pure quaternion and ${\bf w}=s({\rm cos} \frac{\theta}{2}+{\rm sin} \frac{\theta}{2}\boldsymbol{\mu})$ be the quaternion convolution kernel element, where $\theta \in [-\pi,\pi]$ is the rotation angle, $s \in R$ is the scaling factor, and the rotation axis $\boldsymbol{\mu}$ is where $\vec{{\bf p}}$, $\vec{{\bf q}}$ are the vector forms corresponding to the quaternions ${\bf

Figures (6)

  • Figure 1: Schematic of a quaternion generative adversarial network.
  • Figure 2: SVHN Database $16\%$ center Pixel Deletion Comparison of Various Inpainting Methods.
  • Figure 3: Schematic diagram of quaternion batch normalization.
  • Figure 4: SVHN Database $36\%$ center Pixel Deletion Comparison of Various Inpainting Methods.
  • Figure 5: GAN loss map for 1-15000 iterations (with four loss map for iterations 10000-11000, 13000-14000).
  • ...and 1 more figures

Theorems & Definitions (4)

  • Definition 2.1
  • Definition 2.2
  • Theorem 2.1
  • Theorem 3.1