DrawingSpinUp: 3D Animation from Single Character Drawings

TL;DR

DrawingSpinUp tackles the problem of animating a single character drawing in 3D by addressing contour- and texture-ambiguity with a removal-then-restoration pipeline. It combines contour removal via a learned mask $M_c$, coarse 3D proxy generation with a diffusion-prior backbone, geometry refinement including front-view cutting and skeleton-based thinning, and a two-stage geometry-aware stylization network to restore drawing style per frame, followed by rigging and motion retargeting. The approach yields plausible 3D geometries and view-consistent textures, validated by a perceptual user study that shows significant improvements over 2D and other 3D baselines in motion fidelity and style preservation. These contributions enable free-viewpoint animation of amateur drawings, with potential applications in cartoon storytelling, games, and AR/VR content creation, while highlighting limitations around poses, edge artifacts, and abstract drawings for future work.

Abstract

Animating various character drawings is an engaging visual content creation task. Given a single character drawing, existing animation methods are limited to flat 2D motions and thus lack 3D effects. An alternative solution is to reconstruct a 3D model from a character drawing as a proxy and then retarget 3D motion data onto it. However, the existing image-to-3D methods could not work well for amateur character drawings in terms of appearance and geometry. We observe the contour lines, commonly existing in character drawings, would introduce significant ambiguity in texture synthesis due to their view-dependence. Additionally, thin regions represented by single-line contours are difficult to reconstruct (e.g., slim limbs of a stick figure) due to their delicate structures. To address these issues, we propose a novel system, DrawingSpinUp, to produce plausible 3D animations and breathe life into character drawings, allowing them to freely spin up, leap, and even perform a hip-hop dance. For appearance improvement, we adopt a removal-then-restoration strategy to first remove the view-dependent contour lines and then render them back after retargeting the reconstructed character. For geometry refinement, we develop a skeleton-based thinning deformation algorithm to refine the slim structures represented by the single-line contours. The experimental evaluations and a perceptual user study show that our proposed method outperforms the existing 2D and 3D animation methods and generates high-quality 3D animations from a single character drawing. Please refer to our project page (https://lordliang.github.io/DrawingSpinUp) for the code and generated animations.

Figures (14)

• Figure 1: Our DrawingSpinUp produces visually more pleasing character animation results given input drawings and target motions than the existing 2D and 3D animation techniques.
• Figure 2: An example of a character drawing with diverse types of strokes, highlighted in blue (b), red (c) and green (d).
• Figure 3: The pipeline of our DrawingSpinUp. (a) We first remove and inpaint the contour region of the input drawing via an FFC-ResNet. (b) We use a pre-trained Wonder3D to generate a coarse 3D geometry and then refine its shape and texture. (c) We propose a two-stage geometry-aware stylization network to restore the original drawing style (including texture details and contour lines) for each animation frame.
• Figure 4: The process of contour removal.
• Figure 5: An example of surface adhesion.