UniCalli: A Unified Diffusion Framework for Column-Level Generation and Recognition of Chinese Calligraphy

TL;DR

UniCalli introduces a unified diffusion-transformer framework that jointly handles column-level recognition and generation of Chinese calligraphy, addressing both glyph accuracy and page-level layout with global coherence. By modeling latent triplets and spatial priors, and employing strategies like Duplicate RoPE and Conditional Dropout, it achieves state-of-the-art generative fidelity while maintaining competitive recognition, even in limited-data regimes. The approach is validated on a large, annotated dataset and demonstrated to generalize to ancient scripts such as Oracle bone inscriptions and Egyptian hieroglyphs, highlighting significant potential for digital preservation and cross-cultural scholarship. The work also contributes a substantial resource consisting of a large-world calligraphy dataset and public release of code and models, enabling broader research and applications in script analysis and replication.

Abstract

Computational replication of Chinese calligraphy remains challenging. Existing methods falter, either creating high-quality isolated characters while ignoring page-level aesthetics like ligatures and spacing, or attempting page synthesis at the expense of calligraphic correctness. We introduce \textbf{UniCalli}, a unified diffusion framework for column-level recognition and generation. Training both tasks jointly is deliberate: recognition constrains the generator to preserve character structure, while generation provides style and layout priors. This synergy fosters concept-level abstractions that improve both tasks, especially in limited-data regimes. We curated a dataset of over 8,000 digitized pieces, with ~4,000 densely annotated. UniCalli employs asymmetric noising and a rasterized box map for spatial priors, trained on a mix of synthetic, labeled, and unlabeled data. The model achieves state-of-the-art generative quality with superior ligature continuity and layout fidelity, alongside stronger recognition. The framework successfully extends to other ancient scripts, including Oracle bone inscriptions and Egyptian hieroglyphs. Code and data can be viewed in \href{https://github.com/EnVision-Research/UniCalli}{this URL}.

Figures (18)

• Figure 1: Generated by the UniCalli model, this image displays calligraphy from Li Bai's poem "Bring in the Wine". Each column showcases a different master's style to demonstrate the model's versatility. Notably, especially in the Cursive script, the model generates contextually appropriate connecting strokes and character sizes based on adjacent characters. An English translation and the original Chinese text are provided in the lower corners. The complete works by each calligrapher are available in the Appendix\ref{['app:more_results_qjj']}. (The calligraphic background has been manually edited for presentation.)
• Figure 2: An overview of the UniCalli framework. Abbreviations are as follows: Cond. (Condition), Pred. (Prediction), Attn. (Attention), and RoPE (Rotary Positional Embedding).
• Figure 3: Qualitative comparison of our model against state-of-the-art generative models. To facilitate analysis, we use the following visual annotations: a red cross ✗ marks incorrectly rendered glyphs, while a red circle ❍ indicates omitted characters. Furthermore, we highlight desirable calligraphic features: red boxes ❏ showcase well-formed connecting strokes, and blue boxes ❏ emphasize appropriate, context-aware character sizing. Beneath each generated image, we provide style correctness to evaluate its stylistic fidelity.
• Figure 4: Demonstration of UniCalli's robust multi-style calligraphy generation. The same content is rendered in the distinct styles of various calligraphers. The top panel showcases the generation of two Chinese celebrity names, while the bottom panel displays a birthday greeting. This versatility highlights UniCalli's robustness in capturing diverse calligraphic styles and its strong potential for a wide range of downstream applications.
• Figure 5: Ablation study on $p_\text{drop}$. An excessively low $p_\text{drop}$ causes the model to sacrifice structural integrity to replicate long-tail styles, whereas an excessively high value leads to style-agnostic, canonical characters due to over-disentanglement.