UTDesign: A Unified Framework for Stylized Text Editing and Generation in Graphic Design Images

TL;DR

UTDesign addresses the text rendering bottleneck in diffusion-based graphic design by introducing a unified DiT-based framework for high-precision stylized text editing and generation in design images. It combines a disentangled glyph content/style editing backbone with a multi-modal condition encoder and a two-stage layout planner, enabling both editing and conditional generation in multilingual settings. The work introduces SynthGlyph and DesignText datasets to support synthetic style transfer and real design annotations, and culminates in a fully automated text-to-design pipeline that integrates pre-trained text-to-image models and layout planning. Comprehensive experiments show state-of-the-art text rendering accuracy and stylistic consistency among open-source methods, competitive performance against proprietary systems, and strong potential for real-world design workflows. The approach enables practical multilingual glyph editing and automated design generation with end-to-end T2D capabilities.

Abstract

AI-assisted graphic design has emerged as a powerful tool for automating the creation and editing of design elements such as posters, banners, and advertisements. While diffusion-based text-to-image models have demonstrated strong capabilities in visual content generation, their text rendering performance, particularly for small-scale typography and non-Latin scripts, remains limited. In this paper, we propose UTDesign, a unified framework for high-precision stylized text editing and conditional text generation in design images, supporting both English and Chinese scripts. Our framework introduces a novel DiT-based text style transfer model trained from scratch on a synthetic dataset, capable of generating transparent RGBA text foregrounds that preserve the style of reference glyphs. We further extend this model into a conditional text generation framework by training a multi-modal condition encoder on a curated dataset with detailed text annotations, enabling accurate, style-consistent text synthesis conditioned on background images, prompts, and layout specifications. Finally, we integrate our approach into a fully automated text-to-design (T2D) pipeline by incorporating pre-trained text-to-image (T2I) models and an MLLM-based layout planner. Extensive experiments demonstrate that UTDesign achieves state-of-the-art performance among open-source methods in terms of stylistic consistency and text accuracy, and also exhibits unique advantages compared to proprietary commercial approaches. Code and data for this paper are available at https://github.com/ZYM-PKU/UTDesign.

Figures (24)

• Figure 1: Overview of the proposed UTDesign. The first row illustrates the training stages of our model, including: Stage1 (1a): Train from scratch a DiT with content/style encoders to conduct style-preserved text editing; Stage2 (1b): Extract guidance condition from the design background and textual description using MLLM encoder and align the encoded features with the pre-trained style encoder; Stage3 (1c): Replace the style encoder with the MLLM encoder and form a conditional glyph generation model through post-training. The second raw illustrates the detailed structure of the proposed DiT (2a,2b,2c), and shows the training process of our transparency glyph VAE (2d).
• Figure 2: Illustration of some examples of our proposed datasets.
• Figure 3: Comparison of stylized text editing performance with strong baselines. The first column shows original images for selected editing scenarios, with editing targets in the second column. The last three columns present results from three different methods.
• Figure 4: System-level comparison with both open-source and close-source T2D models. We highlight the text rendering problems using red circles.
• Figure 5: User study comparison with proprietary commercial systems.