Local-Global Context-Aware and Structure-Preserving Image Super-Resolution

TL;DR

The paper tackles Real-ISR under diverse real-world degradations by introducing a latent-diffusion framework built on Stable Diffusion with ControlNet conditioning. It proposes Local-Global Context-Aware Attention (LGCAA) to jointly model local pixel relations and long-range dependencies, and a Distribution and Perceptual Aligned Conditioning Module (DPACM) to align latent distributions while preserving perceptual fidelity via Wasserstein and perceptual losses. The approach achieves strong PSNR/SSIM and superior perceptual-quality metrics on RealSR/Real-ISR benchmarks, while mitigating artifacts common in text-guided diffusion methods. By operating in the latent space and leveraging image-based conditioning, the method demonstrates robust restoration that preserves content structure and realistic detail, offering practical improvements for real-world super-resolution tasks. $\\mathcal{L}_{\\mathrm{LGCAA}} = \\mathcal{L}_{\\epsilon} + \lambda_{l}\\mathcal{L}_{\text{perceptual}} + \lambda_{w}\\mathcal{L}_{\text{distribution}}$ encapsulates the joint optimization driving fidelity, perceptual quality, and structural consistency.$

Abstract

Diffusion models have recently achieved significant success in various image manipulation tasks, including image super-resolution and perceptual quality enhancement. Pretrained text-to-image models, such as Stable Diffusion, have exhibited strong capabilities in synthesizing realistic image content, which makes them particularly attractive for addressing super-resolution tasks. While some existing approaches leverage these models to achieve state-of-the-art results, they often struggle when applied to diverse and highly degraded images, leading to noise amplification or incorrect content generation. To address these limitations, we propose a contextually precise image super-resolution framework that effectively maintains both local and global pixel relationships through Local-Global Context-Aware Attention, enabling the generation of high-quality images. Furthermore, we propose a distribution- and perceptual-aligned conditioning mechanism in the pixel space to enhance perceptual fidelity. This mechanism captures fine-grained pixel-level representations while progressively preserving and refining structural information, transitioning from local content details to the global structural composition. During inference, our method generates high-quality images that are structurally consistent with the original content, mitigating artifacts and ensuring realistic detail restoration. Extensive experiments on multiple super-resolution benchmarks demonstrate the effectiveness of our approach in producing high-fidelity, perceptually accurate reconstructions.

Figures (11)

• Figure 1: Comparison of our method with recent state-of-the-art approaches on a degraded image. While existing methods introduce high-frequency details, they often deviate from the original content. In contrast, our method produces high-quality images that maintain a realistic appearance at a global scale while preserving fidelity to the original content when examined closely. This balance between high-frequency and low-frequency information ensures a more natural reconstruction. Please zoom in for a detailed view.
• Figure 2: Comparison of performance and efficiency among Real-ISR methods. For visualization, the metrics LPIPS, DISTS, FID, and NIQE, which are lower-is-better measures of image quality, are inverted and normalized. The proposed method attains superior performance across the majority of evaluated metrics.
• Figure 3: Overall architecture of the proposed LGCAA during training. Here $\mathcal{E}$ is the controlnet conditional embedding and c2D is the 2Dconvolutional block. During training, the model is optimized in the latent space, where the U-Net remains partially frozen with only its attention blocks being trainable. In parallel, the 2D U-Net is conditioned through a trainable ControlNet zhang2023adding, integrated via zero-convolution layers and further augmented with image features extracted from an additional frozen DINO module. During inference, the conditioning part of the Unet and the controlnet is not used instead, the LR is used as input to get the HR image.
• Figure 4: Comparison of our method with other methods on RealSR dataset. It can be seen that our method looks more close to the original ground truth without enhancing any unintended artifacts. Please zoom in for better view.
• Figure 5: Visual comparison of various methods on a real-world dataset example without any HR reference image. Please zoom in for a better view.