MFSR: Multi-fractal Feature for Super-resolution Reconstruction with Fine Details Recovery

TL;DR

Addressing texture-rich detail recovery in single image super-resolution, the paper proposes MFSR, a diffusion-model-based SR framework that leverages multi-fractal features as texture priors. It introduces a MF Feature Extraction Block to approximate fractal information via convolution and an attention-based sub-denoiser to suppress noise during upsampling. Key contributions include the density-estimation-driven MF feature extraction, soft-interval clustering, and an integrated sub-denoiser, with extensive experiments on face and natural image datasets demonstrating improved texture quality over state-of-the-art diffusion-based SR methods. The work highlights the practical potential of fractal-informed conditioning to enhance SR texture fidelity and suggests broader applicability to related vision tasks.

Abstract

In the process of performing image super-resolution processing, the processing of complex localized information can have a significant impact on the quality of the image generated. Fractal features can capture the rich details of both micro and macro texture structures in an image. Therefore, we propose a diffusion model-based super-resolution method incorporating fractal features of low-resolution images, named MFSR. MFSR leverages these fractal features as reinforcement conditions in the denoising process of the diffusion model to ensure accurate recovery of texture information. MFSR employs convolution as a soft assignment to approximate the fractal features of low-resolution images. This approach is also used to approximate the density feature maps of these images. By using soft assignment, the spatial layout of the image is described hierarchically, encoding the self-similarity properties of the image at different scales. Different processing methods are applied to various types of features to enrich the information acquired by the model. In addition, a sub-denoiser is integrated in the denoising U-Net to reduce the noise in the feature maps during the up-sampling process in order to improve the quality of the generated images. Experiments conducted on various face and natural image datasets demonstrate that MFSR can generate higher quality images.

Figures (11)

• Figure 1: This is a demonstration of the results of MFSR's experiments on the FFHQ dataset. Each row represents a sample. The first column represents the input HR image, the second column represents the result of ResDiff, and the third column represents the result of MFSR (Ours). MFSR has better super-resolution effect on details than ResDiff.
• Figure 2: A brief flowchart of the MFSR. Texture features are extracted for LR images processed by pretrained CNNs. Splicing texture features with other features as model input.
• Figure 3: This figure shows 3 low-resolution images and the corresponding super-resolution images. The corresponding multi-fractal spectrum images are shown below the corresponding images and annotated with the corresponding fitted quadratic functions.
• Figure 4: Diagram of the denoiser structure in MFSR. This figure highlights the overall U-Net structure, the multi-fractal analysis module and the sub-denoiser locations. FD Info Spliter reference from resdiff.
• Figure 5: This figure shows a comparison of the multi-fractal calculation process with the one approximated in this paper.