Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

sketch2symm: Symmetry-aware sketch-to-shape generation via semantic bridging

Yan Zhou, Mingji Li, Xiantao Zeng, Jie Lin, Yuexia Zhou

TL;DR

Sketch2Symm tackles the challenge of sketch-based 3D reconstruction by introducing a two-stage framework that first enriches sparse sketches with semantically rich images via sketch-to-image translation and then reconstructs 3D shapes with a symmetry constraint. The method combines a Deformation Alignment Network and Dual-Attention Color Enhancement for semantic bridging, followed by an RGB2Point-based 3D generator with a learned symmetry plane and dual supervision, yielding more complete and regular shapes. Empirical results on ShapeNet-Synthetic and ShapeNet-Sketch show superior Chamfer Distance and Earth Mover's Distance, with favorable F-Score and faster inference than diffusion-based baselines. This approach enables robust, user-friendly sketch-driven 3D modeling with potential applications in VR, robotics, and manufacturing, and opens avenues for handling asymmetric objects in future work.

Abstract

Sketch-based 3D reconstruction remains a challenging task due to the abstract and sparse nature of sketch inputs, which often lack sufficient semantic and geometric information. To address this, we propose Sketch2Symm, a two-stage generation method that produces geometrically consistent 3D shapes from sketches. Our approach introduces semantic bridging via sketch-to-image translation to enrich sparse sketch representations, and incorporates symmetry constraints as geometric priors to leverage the structural regularity commonly found in everyday objects. Experiments on mainstream sketch datasets demonstrate that our method achieves superior performance compared to existing sketch-based reconstruction methods in terms of Chamfer Distance, Earth Mover's Distance, and F-Score, verifying the effectiveness of the proposed semantic bridging and symmetry-aware design.

sketch2symm: Symmetry-aware sketch-to-shape generation via semantic bridging

TL;DR

Sketch2Symm tackles the challenge of sketch-based 3D reconstruction by introducing a two-stage framework that first enriches sparse sketches with semantically rich images via sketch-to-image translation and then reconstructs 3D shapes with a symmetry constraint. The method combines a Deformation Alignment Network and Dual-Attention Color Enhancement for semantic bridging, followed by an RGB2Point-based 3D generator with a learned symmetry plane and dual supervision, yielding more complete and regular shapes. Empirical results on ShapeNet-Synthetic and ShapeNet-Sketch show superior Chamfer Distance and Earth Mover's Distance, with favorable F-Score and faster inference than diffusion-based baselines. This approach enables robust, user-friendly sketch-driven 3D modeling with potential applications in VR, robotics, and manufacturing, and opens avenues for handling asymmetric objects in future work.

Abstract

Sketch-based 3D reconstruction remains a challenging task due to the abstract and sparse nature of sketch inputs, which often lack sufficient semantic and geometric information. To address this, we propose Sketch2Symm, a two-stage generation method that produces geometrically consistent 3D shapes from sketches. Our approach introduces semantic bridging via sketch-to-image translation to enrich sparse sketch representations, and incorporates symmetry constraints as geometric priors to leverage the structural regularity commonly found in everyday objects. Experiments on mainstream sketch datasets demonstrate that our method achieves superior performance compared to existing sketch-based reconstruction methods in terms of Chamfer Distance, Earth Mover's Distance, and F-Score, verifying the effectiveness of the proposed semantic bridging and symmetry-aware design.

Paper Structure

This paper contains 15 sections, 6 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Stage 1: Sketch-to-Image Translation
  4. Stage 2: Image-to-3D Shape Generation
  5. Inference
  6. Experiments
  7. Experimental Settings
  8. Datasets
  9. Evaluation Metrics
  10. Implementation Details
  11. Quantitative Analysis
  12. Qualitative Analysis
  13. Ablation Studies
  14. Comparison with Diffusion-based Methods
  15. Conclusion

Figures (4)

  • Figure 1: The two-stage training pipeline of Sketch2Symm. (a) illustrates the training process of Stage 1 from left to right. (b) shows the training process of Stage 2 from right to left. The bottom left corner shows the details of DACE.
  • Figure 2: Visualization of qualitative comparison on the ShapeNet-Synthetic dataset using synthetic sketches.
  • Figure 3: Visualization of qualitative comparison on the ShapeNet-Sketch dataset using hand-drawn sketches.
  • Figure 4: Qualitative comparison with diffusion-based methods on synthetic and hand-drawn sketches.