TextDiff: Mask-Guided Residual Diffusion Models for Scene Text Image Super-Resolution

TL;DR

TextDiff tackles the persistent challenge of scene text image super-resolution by introducing a diffusion-based framework tailored for text. It combines a Text Enhancement Module (TEM) that jointly deblurs and learns text masks with a Mask-Guided Residual Diffusion Module (MRD) that models the residual between coarse SR outputs and ground-truth text via a conditioned diffusion process, enabling sharp, legible text edges. The approach achieves state-of-the-art recognition performance on TextZoom and delivers strong perceptual quality while enabling plug-and-play improvements to existing STISR methods without additional joint training. This work advances practical STISR by balancing structural fidelity, readability, and computational efficiency through deterministic diffusion steps and targeted text priors.

Abstract

The goal of scene text image super-resolution is to reconstruct high-resolution text-line images from unrecognizable low-resolution inputs. The existing methods relying on the optimization of pixel-level loss tend to yield text edges that exhibit a notable degree of blurring, thereby exerting a substantial impact on both the readability and recognizability of the text. To address these issues, we propose TextDiff, the first diffusion-based framework tailored for scene text image super-resolution. It contains two modules: the Text Enhancement Module (TEM) and the Mask-Guided Residual Diffusion Module (MRD). The TEM generates an initial deblurred text image and a mask that encodes the spatial location of the text. The MRD is responsible for effectively sharpening the text edge by modeling the residuals between the ground-truth images and the initial deblurred images. Extensive experiments demonstrate that our TextDiff achieves state-of-the-art (SOTA) performance on public benchmark datasets and can improve the readability of scene text images. Moreover, our proposed MRD module is plug-and-play that effectively sharpens the text edges produced by SOTA methods. This enhancement not only improves the readability and recognizability of the results generated by SOTA methods but also does not require any additional joint training. Available Codes:https://github.com/Lenubolim/TextDiff.

Figures (71)

• Figure 3: An overview of the proposed TextDiff for scene text image super-resolution. The TEM module consists of two branches, $\textrm{B}_{T}$ and $\textrm{B}_{M}$, and the U-Net structure is the MRD module.
• Figure 4: The SR image results on TextZoom.
• Figure 6: Qualitative ablation study results on TextZoom. “w/o” denotes without, “NP” denotes “Noise Prediction”, “NU” denotes an equivalent U-Net is used in place of a diffusion model.
• Figure : TSRN
• Figure : LR mask