CLIP-aware Domain-Adaptive Super-Resolution

TL;DR

CDASR tackles cross-domain single image super-resolution by fusing CLIP-derived semantic features with SR representations through a dedicated domain-adaptive module, complemented by a meta-learning inspired few-shot adaptation strategy. The framework employs a CLIP-guided feature alignment and a multi-component loss to preserve pixel fidelity and high-level semantics across diverse domains and extreme scaling factors. Theoretical analysis provides a CLIP-guided generalization bound, while extensive experiments demonstrate state-of-the-art performance on benchmarks such as Urban100 and Manga109, especially at ×8–×16 scales, with notable improvements under limited target-domain supervision. This approach offers practical benefits for real-world SR tasks where domain shifts are pronounced, enabling rapid adaptation with minimal labeled data and robust semantic preservation.

Abstract

This work introduces CLIP-aware Domain-Adaptive Super-Resolution (CDASR), a novel framework that addresses the critical challenge of domain generalization in single image super-resolution. By leveraging the semantic capabilities of CLIP (Contrastive Language-Image Pre-training), CDASR achieves unprecedented performance across diverse domains and extreme scaling factors. The proposed method integrates CLIP-guided feature alignment mechanism with a meta-learning inspired few-shot adaptation strategy, enabling efficient knowledge transfer and rapid adaptation to target domains. A custom domain-adaptive module processes CLIP features alongside super-resolution features through a multi-stage transformation process, including CLIP feature processing, spatial feature generation, and feature fusion. This intricate process ensures effective incorporation of semantic information into the super-resolution pipeline. Additionally, CDASR employs a multi-component loss function that combines pixel-wise reconstruction, perceptual similarity, and semantic consistency. Extensive experiments on benchmark datasets demonstrate CDASR's superiority, particularly in challenging scenarios. On the Urban100 dataset at $\times$8 scaling, CDASR achieves a significant PSNR gain of 0.15dB over existing methods, with even larger improvements of up to 0.30dB observed at $\times$16 scaling.

Figures (10)

• Figure 1: 2D t-SNE visualization of CLIP features extracted from various super-resolution benchmark datasets. DIV2K (blue) shows a wide distribution covering multiple domains, BSD100 (red) clusters predominantly in one region with partial overlap with DIV2K, while Manga109 (purple) forms a distinctly separate cluster due to its unique anime-style content. This distribution highlights the significant domain differences that super-resolution models must address.
• Figure 2: Overview of the proposed CDASR framework. The CLIP-guided alignment module (a) fuses semantic information from CLIP with SR features, while the reconstruction module (b) generates the final high-resolution output.
• Figure 3: CLIP-guided alignment module fusing semantic and spatial information for enhanced super-resolution.
• Figure 4: Reconstruction module architecture for generating high-resolution output from aligned features.
• Figure 5: Visual comparison of super-resolution results ($\times4$) on the butterfly image from Set5 dataset. The red box highlights the detailed region shown across methods. PSNR/SSIM values beneath each result demonstrate CDASR's superior performance (22.42/0.8510) in preserving the intricate butterfly wing patterns.