Versatile Cataract Fundus Image Restoration Model Utilizing Unpaired Cataract and High-quality Images

TL;DR

This work tackles cataract-induced degradation in fundus images by introducing Catintell, a two-stage framework with Catintell-Syn for unsupervised synthesis of cataract-like images and Catintell-Res for restoration trained on synthetic pairs. Catintell-Res combines a CNN-based generator with a Dense Conv Block encoder and a Swin-Transformer discriminator, guided by Fundus Perceptual Loss and other losses to achieve high-fidelity restoration. The method demonstrates strong quantitative performance (PSNR $39.03$, SSIM $0.9476$ on synthetic data) and robust generalization to real cataract datasets, supported by ophthalmologist user studies that favor Catintell-Res versus state-of-the-art baselines. The results indicate potential for improved preoperative diagnosis and broader applicability to medical image restoration, with plans to release the models for research and clinical use.

Abstract

Cataract is one of the most common blinding eye diseases and can be treated by surgery. However, because cataract patients may also suffer from other blinding eye diseases, ophthalmologists must diagnose them before surgery. The cloudy lens of cataract patients forms a hazy degeneration in the fundus images, making it challenging to observe the patient's fundus vessels, which brings difficulties to the diagnosis process. To address this issue, this paper establishes a new cataract image restoration method named Catintell. It contains a cataract image synthesizing model, Catintell-Syn, and a restoration model, Catintell-Res. Catintell-Syn uses GAN architecture with fully unsupervised data to generate paired cataract-like images with realistic style and texture rather than the conventional Gaussian degradation algorithm. Meanwhile, Catintell-Res is an image restoration network that can improve the quality of real cataract fundus images using the knowledge learned from synthetic cataract images. Extensive experiments show that Catintell-Res outperforms other cataract image restoration methods in PSNR with 39.03 and SSIM with 0.9476. Furthermore, the universal restoration ability that Catintell-Res gained from unpaired cataract images can process cataract images from various datasets. We hope the models can help ophthalmologists identify other blinding eye diseases of cataract patients and inspire more medical image restoration methods in the future.

Figures (8)

• Figure 1: Catintell Model Workflow. We use two GAN models to generate synthetic cataract images and restore cataract images separately. The idea is to collect the information contained in real cataract images and let Catintell-Syn learn from it. Then Catintell-Res learns from synthetic data generated by Catintell-Syn and works on real cataract images from various datasets. Existing methods focus on learning from synthetic data generated by an old method41493, which may not contain the features of real cataract images. But Catintell extracts features directly from real cataract images and applies them to real cataract image restoration.
• Figure 2: The structure of the Catintell model. (a) The example model has a four-stage convolutional generator with downsampling and upsampling multiplier 2. (b) The discriminator of Catintell is a Transformer-based classifier and has four stages. (c) Detailed structure of the Dense Conv Block.
• Figure 3: Sample of our Catintell Image dataset. (a) 2436 cataract images were collected in this dataset. (b) 1144 high-quality images were collected.
• Figure 4: Result of degraded images from Catintell-Syn and traditional modeling method. (a) Source HQ fundus images. (b) Synthetic cataract fundus images using traditional method. (c) using CycleGAN. (d) using Catintell-Syn. (d) Real cataract fundus image samples. The images generated by Catintell-Syn are more similar to real cataract fundus images.
• Figure 5: Restored real cataract image comparisons of Scene 1 on a test image of the Catintell Image dataset. Compared to other methods, the vessels around the macula in the restored image of Catintell-Res are finely enhanced. The overall style of this image is also maintained rather than changed to a dark/orange color.