Textoon: Generating Vivid 2D Cartoon Characters from Text Descriptions

TL;DR

Textoon tackles the challenge of efficiently generating interactive 2D Live2D characters from natural text. It combines a fine-tuned large language model for parsing component-level descriptions, a controllable SDXL-based appearance generator, and a re-editing and completion pipeline to assemble accurate Live2D assets. The key novelty lies in refactoring Live2D facial animation with ARKit-based blendshapes and a speech-driven network using Wav2Vec-XLSR to produce richer lip-sync, all rendered in lightweight HTML5. The approach enables creation of diverse, drivable 2D characters in under a minute, broadening accessible, interactive 2D avatars for mobile and web platforms.

Abstract

The 2D cartoon style is a prominent art form in digital character creation, particularly popular among younger audiences. While advancements in digital human technology have spurred extensive research into photorealistic digital humans and 3D characters, interactive 2D cartoon characters have received comparatively less attention. Unlike 3D counterparts, which require sophisticated construction and resource-intensive rendering, Live2D, a widely-used format for 2D cartoon characters, offers a more efficient alternative, which allows to animate 2D characters in a manner that simulates 3D movement without the necessity of building a complete 3D model. Furthermore, Live2D employs lightweight HTML5 (H5) rendering, improving both accessibility and efficiency. In this technical report, we introduce Textoon, an innovative method for generating diverse 2D cartoon characters in the Live2D format based on text descriptions. The Textoon leverages cutting-edge language and vision models to comprehend textual intentions and generate 2D appearance, capable of creating a wide variety of stunning and interactive 2D characters within one minute. The project homepage is https://human3daigc.github.io/Textoon_webpage/.

Figures (12)

• Figure 1: Examples of animatable 2D cartoon characters generated by Textoon.
• Figure 2: Pipeline of the Textoon. The framework includes text parsing, controllable appearance generation, re-editing, and component completion and repair modules.
• Figure 3: Meshes of different layers.
• Figure 4: Splitting model components, larger elements can be utilized to create short variations.
• Figure 5: Using the fine-tuned LLM to parse component categories from complex input text.