GLYPH-SR: Can We Achieve Both High-Quality Image Super-Resolution and High-Fidelity Text Recovery via VLM-guided Latent Diffusion Model?

TL;DR

GLYPH-SR tackles the challenge of image super-resolution in text-rich scenes by explicitly optimizing for both visual fidelity and text legibility. It introduces a vision–language guided diffusion framework with a dual-branch TS-ControlNet that fuses OCR-derived glyph cues and a global scene caption, augmented by a ping-pong scheduler that alternates between text-focused and image-focused guidance. A factorized synthetic corpus enables targeted text restoration while freezing the main SR branch, and a dual-axis evaluation pairs OCR accuracy with perceptual SR metrics to ensure balanced performance. On challenging benchmarks across scales, GLYPH-SR delivers substantial OCR F1 improvements (up to +15.18 pp) while maintaining competitive perceptual quality, demonstrating a practical pathway to readable and realistic super-resolved imagery.

Abstract

Image super-resolution(SR) is fundamental to many vision system-from surveillance and autonomy to document analysis and retail analytics-because recovering high-frequency details, especially scene-text, enables reliable downstream perception. Scene-text, i.e., text embedded in natural images such as signs, product labels, and storefronts, often carries the most actionable information; when characters are blurred or hallucinated, optical character recognition(OCR) and subsequent decisions fail even if the rest of the image appears sharp. Yet previous SR research has often been tuned to distortion (PSNR/SSIM) or learned perceptual metrics (LIPIS, MANIQA, CLIP-IQA, MUSIQ) that are largely insensitive to character-level errors. Furthermore, studies that do address text SR often focus on simplified benchmarks with isolated characters, overlooking the challenges of text within complex natural scenes. As a result, scene-text is effectively treated as generic texture. For SR to be effective in practical deployments, it is therefore essential to explicitly optimize for both text legibility and perceptual quality. We present GLYPH-SR, a vision-language-guided diffusion framework that aims to achieve both objectives jointly. GLYPH-SR utilizes a Text-SR Fusion ControlNet(TS-ControlNet) guided by OCR data, and a ping-pong scheduler that alternates between text- and scene-centric guidance. To enable targeted text restoration, we train these components on a synthetic corpus while keeping the main SR branch frozen. Across SVT, SCUT-CTW1500, and CUTE80 at x4, and x8, GLYPH-SR improves OCR F1 by up to +15.18 percentage points over diffusion/GAN baseline (SVT x8, OpenOCR) while maintaining competitive MANIQA, CLIP-IQA, and MUSIQ. GLYPH-SR is designed to satisfy both objectives simultaneously-high readability and high visual realism-delivering SR that looks right and reds right.

Figures (6)

• Figure 1: Qualitative and quantitative comparisons of our GLYPH-SR with other competing SR methods, demonstrating superior text fidelity and OCR F1 score.
• Figure 2: Overview of the proposed GLYPH-SR architecture.
• Figure 3: Text-centric fine-tuning framework: (a) trade-off between scene-text fidelity and overall image quality according to guidance; (b) four synthetic training subsets with matched prompts; (c) TS-ControlNet architecture.
• Figure 4: Qualitative examples illustrating the trade-off between SR metrics (e.g., MANIQA, CLIP-IQA, MUSIQ) and OCR metrics (F1, Accuracy) in scene-text images. While some methods improve perceptual SR scores, they may degrade OCR performance, and vice versa.
• Figure 5: Comparison of SR results against different methods (DiffBIR, Real-ESRGAN, BSRGAN, and GLYPH-SR) on various degraded LR images.