Detail-Enhancing Framework for Reference-Based Image Super-Resolution

TL;DR

This paper addresses misalignment and texture-mismatch challenges in reference-based image super-resolution by introducing a Detail-Enhancing Framework (DEF) that uses a pre-trained diffusion model to generate and stabilize fine details in the low-resolution input before alignment with reference textures. The approach combines diffusion-based detail generation with a standard Ref-SR pipeline, leveraging range-null space decomposition to maintain data consistency while enriching realism. It employs multi-scale feature extraction, cosine-based index and confidence maps, and deformable convolutions for robust texture transfer and integration, along with cross-scale aggregation to fuse textures effectively. Experiments on CUFED5 and five test sets show that DEF yields superior visual quality and competitive PSNR/SSIM, validating the method's ability to reduce texture mismatch and improve alignment, particularly when corresponding textures are present in references. The work offers practical impact for high-fidelity SR in scenarios with imperfect references and varying textures, enabling more reliable detail restoration and fewer artifacts in real-world applications.

Abstract

Recent years have witnessed the prosperity of reference-based image super-resolution (Ref-SR). By importing the high-resolution (HR) reference images into the single image super-resolution (SISR) approach, the ill-posed nature of this long-standing field has been alleviated with the assistance of texture transferred from reference images. Although the significant improvement in quantitative and qualitative results has verified the superiority of Ref-SR methods, the presence of misalignment before texture transfer indicates room for further performance improvement. Existing methods tend to neglect the significance of details in the context of comparison, therefore not fully leveraging the information contained within low-resolution (LR) images. In this paper, we propose a Detail-Enhancing Framework (DEF) for reference-based super-resolution, which introduces the diffusion model to generate and enhance the underlying detail in LR images. If corresponding parts are present in the reference image, our method can facilitate rigorous alignment. In cases where the reference image lacks corresponding parts, it ensures a fundamental improvement while avoiding the influence of the reference image. Extensive experiments demonstrate that our proposed method achieves superior visual results while maintaining comparable numerical outcomes.

Figures (3)

• Figure 1: In the existing Ref-SR methods, such as TTSR RN21, performance often deteriorates due to misalignment. To address this issue, we propose enhancing the fine-grained textures within images using a pre-trained diffusion model, thereby aiding the alignment process.
• Figure 2: Detail-Enhancing Framework overview. For the input LR image, we commence by subjecting it to a diffusion process to enhance its fine details. Subsequently, both the detail-enhanced image and the reference image undergo feature extraction through a structurally identical network. The extracted features are aligned to obtain an index map and a confidence map, serving as the basis for the final multi-scale aggregation process.
• Figure 3: Visual comparison with other methods. We zoom in on the key areas for a better view.