Multi-Scale Target-Aware Representation Learning for Fundus Image Enhancement

TL;DR

This work proposes a multi-scale target-aware representation learning framework (MTRL-FIE), which achieves superior enhancement performance with a more lightweight architecture and generalizes to other ophthalmic image processing tasks without supervised fine-tuning, highlighting its potential for clinical applications.

Abstract

High-quality fundus images provide essential anatomical information for clinical screening and ophthalmic disease diagnosis. Yet, due to hardware limitations, operational variability, and patient compliance, fundus images often suffer from low resolution and signal-to-noise ratio. Recent years have witnessed promising progress in fundus image enhancement. However, existing works usually focus on restoring structural details or global characteristics of fundus images, lacking a unified image enhancement framework to recover comprehensive multi-scale information. Moreover, few methods pinpoint the target of image enhancement, e.g., lesions, which is crucial for medical image-based diagnosis. To address these challenges, we propose a multi-scale target-aware representation learning framework (MTRL-FIE) for efficient fundus image enhancement. Specifically, we propose a multi-scale feature encoder (MFE) that employs wavelet decomposition to embed both low-frequency structural information and high-frequency details. Next, we design a structure-preserving hierarchical decoder (SHD) to fuse multi-scale feature embeddings for real fundus image restoration. SHD integrates hierarchical fusion and group attention mechanisms to achieve adaptive feature fusion while retaining local structural smoothness. Meanwhile, a target-aware feature aggregation (TFA) module is used to enhance pathological regions and reduce artifacts. Experimental results on multiple fundus image datasets demonstrate the effectiveness and generalizability of MTRL-FIE for fundus image enhancement. Compared to state-of-the-art methods, MTRL-FIE achieves superior enhancement performance with a more lightweight architecture. Furthermore, our approach generalizes to other ophthalmic image processing tasks without supervised fine-tuning, highlighting its potential for clinical applications.

Figures (8)

• Figure 1: Overview of the proposed MTRL-FIE. The MFE maps the degraded fundus image into a wavelet feature pyramid. The SHD reconstructs two complementary streams: fine-grained, high-frequency details and large-scale anatomical structures. The TFA applies spatial–channel attention to select and fuse the most relevant signals from both streams to produce the final reconstruction.
• Figure 2: The workflow of the MTRL-FIE framework. First, a high-quality image $I$ is degraded using a transformation function $D(I)$ to generate a low-quality input. The degraded image is processed by a multi-level encoder that progressively downsamples the input, generating an embedded representation enriched with multi-level wavelet features. This representation is then upsampled by a series of high-frequency convolutional decoders, and supervision loss is computed by comparing the recovered and original high-frequency components. At each resolution scale, an attention-based fusion module combines the wavelet-encoded features with the upsampled high-frequency representations, further enhancing fine details. Finally, the high-frequency-enhanced embedding, which integrates multi-level wavelet features, is used to reconstruct the high-quality image.
• Figure 3: The high-frequency enhanced encoder decomposes the input image into multiple hierarchical frequency sub-bands. Using downsampling and wavelet transforms, the MFE extracts four components: approximation, vertical, horizontal, and diagonal. These sub-band features are refined through depthwise separable convolutions and then reconstructed via inverse wavelet transform, enabling robust embedding of both coarse and fine structural details. A residual connection, implemented with depthwise separable convolutions, links the output back to the input. This prepares the multi-scale features for attention-based fusion and high-frequency enhancement in the decoding stage.
• Figure 4: The unified attention fusion module combines spatial and channel information using spatial average pooling, along with channel-wise mean and max operations. A bottleneck structure followed by convolutional layers further enhances feature selectivity. The SCF module refines the fused features, emphasizing important high-frequency components for improved decoding performance. The high-frequency convolutional decoder progressively upsamples the multi-scale wavelet features. Leveraging group attention and depthwise separable convolutions, the SHD reconstructs high-quality images with minimal parameter overhead, while effectively preserving fine-grained, high-frequency details.
• Figure 5: Images synthesized on the BA dataset using different degradation methods. (a) Original high-quality image; (b) Image degraded with brightness and saturation adjustments; (c) Image with added circular spot noise; (d) Image affected by simulated cataract degradation; (e) Image with applied Gaussian blur; (f) Image with multiple combined degradation types.