HDGlyph: A Hierarchical Disentangled Glyph-Based Framework for Long-Tail Text Rendering in Diffusion Models

TL;DR

HDGlyph tackles long-tail text rendering in diffusion models by hierarchical disentanglement across pixel, noise, and latent levels, leveraging a Multi-Linguistic GlyphNet with language-specific LoRAs, a Glyph-Aware Perceptual Loss, Noise-Disentangled Classifier-Free Guidance, and Latent-Disentangled Two-Stage Rendering. It achieves significant improvements in text accuracy for English and Chinese while maintaining high image quality on benchmarks including AnyText, Multilingual, and Complex, particularly in unseen characters and small-sized text. The approach demonstrates strong generalization for multilingual and script-dense content, enabling practical applications in design, localization, and content generation. Collectively, this work establishes a scalable framework for universal visual text rendering in diffusion models.

Abstract

Visual text rendering, which aims to accurately integrate specified textual content within generated images, is critical for various applications such as commercial design. Despite recent advances, current methods struggle with long-tail text cases, particularly when handling unseen or small-sized text. In this work, we propose a novel Hierarchical Disentangled Glyph-Based framework (HDGlyph) that hierarchically decouples text generation from non-text visual synthesis, enabling joint optimization of both common and long-tail text rendering. At the training stage, HDGlyph disentangles pixel-level representations via the Multi-Linguistic GlyphNet and the Glyph-Aware Perceptual Loss, ensuring robust rendering even for unseen characters. At inference time, HDGlyph applies Noise-Disentangled Classifier-Free Guidance and Latent-Disentangled Two-Stage Rendering (LD-TSR) scheme, which refines both background and small-sized text. Extensive evaluations show our model consistently outperforms others, with 5.08% and 11.7% accuracy gains in English and Chinese text rendering while maintaining high image quality. It also excels in long-tail scenarios with strong accuracy and visual performance.

Figures (4)

• Figure 1: Illustration of our motivation. The curve chart above demonstrates the limitations of existing models on long-tail text. (a) Under the same font size, image quality decreases as the rarity of the text in the training dataset increases. (b) Under similar image quality, text accuracy decreases as the rarity of the text in the training dataset increases, and increases with the font size. For further details, refer to Appendix A. (c) Shows the hierarchical disentangling concept of HDGlyph that we propose.
• Figure 2: Glyph-aware Training Pipeline of HDGlyph.Blue modules are frozen, red modules are trainable, and white modules are styled differently to distinguish them from the U-Net.
• Figure 3: Our HDGlyph framework inference pipeline comprises Multi-Linguistic GlyphNet, along with (a) the Noise-Disentangled Classifier-Free Guidance (ND-CFG) module for improving glyph representation and (b.1) the Latent-Disentangled Two-Stage Rendering (LD-TSR) module for spatially separating text from the background to enhance image quality; and (b.2) the latent-disentanglement for small text rendering, which enables finer-grained glyph control at the latent level. It is noteworthy that we have omitted the process of decoding the noise image from the latent space.
• Figure 4: Qualitative comparison of HDGlyph with state-of-the-art models in long-tail text rendering of multilingual and small-sized text.