4-Doodle: Text to 3D Sketches that Move!

TL;DR

This paper tackles the problem of transforming natural language into dynamic 3D sketches, a challenging task due to the absence of paired text–4D data and the need for view-consistent, abstract representations. It introduces 4-Doodle, a training-free framework that religiously distills priors from image and video diffusion models into a dual-space architecture: a structure space based on differentiable Bézier curves for multi-view geometry, and a motion space that learns 3D displacement fields for animation. The method proceeds in two stages—Stage I enforces multi-view geometric coherence, while Stage II learns temporally consistent motion via projection-reconstruction with video priors—yielding temporally realistic 3D sketch animations. The authors demonstrate superior qualitative and quantitative performance over baselines, identify limitations in existing CLIP-based metrics for 3D sketches, and propose an improved evaluation framework using a vision-language model to better capture structure, motion, and abstraction in 4D content creation.

Abstract

We present a novel task: text-to-3D sketch animation, which aims to bring freeform sketches to life in dynamic 3D space. Unlike prior works focused on photorealistic content generation, we target sparse, stylized, and view-consistent 3D vector sketches, a lightweight and interpretable medium well-suited for visual communication and prototyping. However, this task is very challenging: (i) no paired dataset exists for text and 3D (or 4D) sketches; (ii) sketches require structural abstraction that is difficult to model with conventional 3D representations like NeRFs or point clouds; and (iii) animating such sketches demands temporal coherence and multi-view consistency, which current pipelines do not address. Therefore, we propose 4-Doodle, the first training-free framework for generating dynamic 3D sketches from text. It leverages pretrained image and video diffusion models through a dual-space distillation scheme: one space captures multi-view-consistent geometry using differentiable Bézier curves, while the other encodes motion dynamics via temporally-aware priors. Unlike prior work (e.g., DreamFusion), which optimizes from a single view per step, our multi-view optimization ensures structural alignment and avoids view ambiguity, critical for sparse sketches. Furthermore, we introduce a structure-aware motion module that separates shape-preserving trajectories from deformation-aware changes, enabling expressive motion such as flipping, rotation, and articulated movement. Extensive experiments show that our method produces temporally realistic and structurally stable 3D sketch animations, outperforming existing baselines in both fidelity and controllability. We hope this work serves as a step toward more intuitive and accessible 4D content creation.

Figures (15)

• Figure 1: Given a text description, a dynamic 3D vector sketch can be generated by our model. Depth cues are colour-coded.
• Figure 2: Overview of 4-Doodle: text-driven dynamic 3D sketch generation. Stage I: Multi-view 3D sketch optimization using SDS loss from randomly initialized Bézier curves. Stage II: Motion field learning through projection-reconstruction strategy that decomposes 3D scenes into front/side views for video SDS optimization.
• Figure 3: Text-to-4D Sketch Generation. Generated 4D sketches from text prompts showing motion sequences and multi-object scenes with temporal dynamics and sketch-style representation.
• Figure 4: Comparison results of text-to-3D (in different viewpoints) and sketch animation.
• Figure 5: Training with different number of views and effects of guidance strength and number of strokes.