R2C-GAN: Restore-to-Classify Generative Adversarial Networks for Blind X-Ray Restoration and COVID-19 Classification

TL;DR

The paper tackles blind X-ray restoration under mixed, unknown artifacts while preserving disease information for COVID-19 classification. It introduces Restore-to-Classify GANs (R2C-GANs) that learn forward and inverse mappings between poor and high-quality X-ray domains ($G: Y \rightarrow X$ and $F: X \rightarrow Y$) while jointly predicting class labels, guided by adversarial, cycle-consistency, identity, and classification losses. By embedding Self-Organized Operational Neural Networks, the authors build compact generators that maintain high performance, demonstrated on the QaTa-COV19 dataset where COVID-19 classification achieves state-of-the-art results and clinician-validated restorations are reported. The work shows that task-driven restoration can improve downstream diagnosis and segmentation, and the framework can be extended to other biomedical signals requiring robust, artifact-agnostic restoration.

Abstract

Restoration of poor quality images with a blended set of artifacts plays a vital role for a reliable diagnosis. Existing studies have focused on specific restoration problems such as image deblurring, denoising, and exposure correction where there is usually a strong assumption on the artifact type and severity. As a pioneer study in blind X-ray restoration, we propose a joint model for generic image restoration and classification: Restore-to-Classify Generative Adversarial Networks (R2C-GANs). Such a jointly optimized model keeps any disease intact after the restoration. Therefore, this will naturally lead to a higher diagnosis performance thanks to the improved X-ray image quality. To accomplish this crucial objective, we define the restoration task as an Image-to-Image translation problem from poor quality having noisy, blurry, or over/under-exposed images to high quality image domain. The proposed R2C-GAN model is able to learn forward and inverse transforms between the two domains using unpaired training samples. Simultaneously, the joint classification preserves the disease label during restoration. Moreover, the R2C-GANs are equipped with operational layers/neurons reducing the network depth and further boosting both restoration and classification performances. The proposed joint model is extensively evaluated over the QaTa-COV19 dataset for Coronavirus Disease 2019 (COVID-19) classification. The proposed restoration approach achieves over 90% F1-Score which is significantly higher than the performance of any deep model. Moreover, in the qualitative analysis, the restoration performance of R2C-GANs is approved by a group of medical doctors. We share the software implementation at https://github.com/meteahishali/R2C-GAN.

Figures (8)

• Figure 1: The proposed approach learns forward and inverse transformations between poor and high quality X-ray image domains. During the transitions, disease traces are aimed to be preserved by predicting the class labels.
• Figure 2: The R2C-GAN framework is divided into two parts for the illustration purposes. There are two generator networks providing $G: Y \rightarrow X, C_Y$ and $F: X \rightarrow Y, C_X$ where $Y$ and $X$ are poor and high quality image domains, respectively, with $\mathbf{y} \in Y$, $\mathbf{x} \in X$. Their class labels are $\mathbf{c}_{y} \in C_Y$ and $\mathbf{c}_{x} \in C_X$ where $C_Y, C_X \in \mathbb{R}^{N_C}$ with $N_C$ number of classes. Two discriminators learn the following mappings: $D_X: X \rightarrow M_X$ and $D_Y: Y \rightarrow M_Y$ where $M_X, M_Y \in \mathbb{R}^{d_m \times d_n}$ are the predicted masks.
• Figure 3: The R2C-GANs have the presented compact and novel structure for their generator networks $G$ and $F$. The proposed generator model applies two tasks in a single inference: Image-to-Image translation and classification.
• Figure 4: The proposed compact and novel network structure is illustrated for the discriminator networks $D_X$ and $D_Y$ in R2C-GANs.
• Figure 5: Poor quality (a) and high quality (b) image samples from the QaTa-COV19 dataset.