3Doodle: Compact Abstraction of Objects with 3D Strokes

TL;DR

3Doodle introduces a compact, differentiable framework to generate view-consistent 2D sketches from multi-view images by representing objects with a small set of 3D primitives: view-independent 3D cubic Bézier curves and view-dependent contours from superquadrics. The method renders these primitives through a fully differentiable pipeline and optimizes their parameters directly against perceptual losses (LPIPS and CLIP) without requiring explicit 3D meshes or NeRF training. Key contributions include the first 3D-stroke based sketch generation from multi-view observations, an extremely compact representation (<1.5 kB), and demonstrated ability to capture essential 3D structure across diverse objects with view coherence. This approach enables scalable sketch-based representation and has potential applications in dataset creation, educational visualization, and aiding 3D reconstruction tasks.

Abstract

While free-hand sketching has long served as an efficient representation to convey characteristics of an object, they are often subjective, deviating significantly from realistic representations. Moreover, sketches are not consistent for arbitrary viewpoints, making it hard to catch 3D shapes. We propose 3Dooole, generating descriptive and view-consistent sketch images given multi-view images of the target object. Our method is based on the idea that a set of 3D strokes can efficiently represent 3D structural information and render view-consistent 2D sketches. We express 2D sketches as a union of view-independent and view-dependent components. 3D cubic B ezier curves indicate view-independent 3D feature lines, while contours of superquadrics express a smooth outline of the volume of varying viewpoints. Our pipeline directly optimizes the parameters of 3D stroke primitives to minimize perceptual losses in a fully differentiable manner. The resulting sparse set of 3D strokes can be rendered as abstract sketches containing essential 3D characteristic shapes of various objects. We demonstrate that 3Doodle can faithfully express concepts of the original images compared with recent sketch generation approaches.

Figures (14)

• Figure 1: Overview of 3Doodle. We generate a compact 3D geometric representation from multi-view images of objects. We separately define view-independent stroke ($\mathcal{S}^{\text{3D}}_{\text{ind}}$) and view-dependent stroke ($\mathcal{S}^{\text{3D}}_{\text{dep}}$). We represent a view-independent stroke as a set of 3D Bézier curves and a view-dependent stroke as a contour of superquadrics. (Sec. 3.1) We also propose a fully differentiable rendering method to render sketches from the 3D strokes. (Sec. 3.2) Finally, our 3D stroke parameters are directly optimized with perceptual losses. (Sec. 3.3)
• Figure 2: (a) We visualize the volume density $\sigma_{\text{vol}}$ and surface volume density $\sigma_{\text{surf}}$. (b) We display the volume-rendered results of each volume density component. One can obtain the contour of geometric volume by volume rendering our proposed view-dependent contour volume density $\sigma_{\text{contour}}$.
• Figure 3: Bar plots of the perceptual study results.
• Figure 4: Qualitative results. We show the target objects in the leftmost column and multi-view rendered results of our 3D strokes in the right three columns.
• Figure 5: 3Doodle robustly generates sketch for a few multi-view inputs.

Theorems & Definitions (5)

• Theorem 1
• Theorem 1
• proof
• Theorem 2
• proof