Text-Conditioned Diffusion Model for High-Fidelity Korean Font Generation

TL;DR

This work tackles high-fidelity Korean font generation from a single reference image by introducing DK-Font, a diffusion-based, one-shot AFG framework. It integrates a text-conditioned encoder to capture phonetic content, an enhanced attribute encoder to fuse content, style, and stroke information, and perceptual loss to preserve global structure, leveraging a pre-trained DG FONT style encoder for accurate style transfer. Trained on a large set of 2,350 Korean characters and evaluated against Diff-Font, DK-Font achieves higher SSIM and lower RMSE, LPIPS, and FID across handwritten and printed fonts, including unseen characters. The approach demonstrates robust, diverse, and authentic Korean font generation with practical potential for scalable font design.

Abstract

Automatic font generation (AFG) is the process of creating a new font using only a few examples of the style images. Generating fonts for complex languages like Korean and Chinese, particularly in handwritten styles, presents significant challenges. Traditional AFGs, like Generative adversarial networks (GANs) and Variational Auto-Encoders (VAEs), are usually unstable during training and often face mode collapse problems. They also struggle to capture fine details within font images. To address these problems, we present a diffusion-based AFG method which generates high-quality, diverse Korean font images using only a single reference image, focusing on handwritten and printed styles. Our approach refines noisy images incrementally, ensuring stable training and visually appealing results. A key innovation is our text encoder, which processes phonetic representations to generate accurate and contextually correct characters, even for unseen characters. We used a pre-trained style encoder from DG FONT to effectively and accurately encode the style images. To further enhance the generation quality, we used perceptual loss that guides the model to focus on the global style of generated images. Experimental results on over 2000 Korean characters demonstrate that our model consistently generates accurate and detailed font images and outperforms benchmark methods, making it a reliable tool for generating authentic Korean fonts across different styles.

Figures (4)

• Figure 1: Overview of the Diffusion Model Process. The forward diffusion process (top row) transforms the original image into a noisy representation by gradually adding noise. The reverse process (bottom row) uses a deep learning model to progressively remove the noise, ultimately reconstructing the original data. The enlarged section on the right illustrates a single reverse step, where noise is estimated and removed from the current state $x_t$ to approximate the previous state $x_{t-1}$.
• Figure 2: The framework of our model refines noisy input to generate font images. An Enhanced Character Attributes Encoder integrates strokes, components, and style features with text embeddings to guide this process.
• Figure 3: Comparison of our method with Diff-Font and ground truth on handwritten Korean fonts.
• Figure 4: Comparison of our method with Diff-Font and ground truth on printed fonts.