Img2CAD: Conditioned 3D CAD Model Generation from Single Image with Structured Visual Geometry

TL;DR

Img2CAD, the first approach to the authors' knowledge that uses 2-D image inputs to generate computer-aided design (CAD) models with editable parameters, is proposed and an innovative intermediate representation called structured visual geometry is identified, characterized by vectorized wireframes extracted from objects.

Abstract

In this paper, we propose Img2CAD, the first approach to our knowledge that uses 2D image inputs to generate CAD models with editable parameters. Unlike existing AI methods for 3D model generation using text or image inputs often rely on mesh-based representations, which are incompatible with CAD tools and lack editability and fine control, Img2CAD enables seamless integration between AI-based 3D reconstruction and CAD software. We have identified an innovative intermediate representation called Structured Visual Geometry (SVG), characterized by vectorized wireframes extracted from objects. This representation significantly enhances the performance of generating conditioned CAD models. Additionally, we introduce two new datasets to further support research in this area: ABC-mono, the largest known dataset comprising over 200,000 3D CAD models with rendered images, and KOCAD, the first dataset featuring real-world captured objects alongside their ground truth CAD models, supporting further research in conditioned CAD model generation.

Figures (11)

• Figure 1: We propose the first 3D model generation network that can produce “sketch and extrude" parametric command representation of 3D objects with only single images or sketches as the input. (a) Examples generated from the images or sketches. (b) More examples of CAD designs were obtained from our Img2CAD approach, which can be used as the coarse rapid prototyping stage for 3D modeling by experts to make the modeling process faster.
• Figure 2: The Limitation of Existing 3D AIGC Approach in Generating Simple Man-Made Geometry with Image Conditions. Our method aims to generate high-quality man-made objects guided by image conditions using CAD representation instead of using mesh, which allows high-quality surface creation and direct integration with traditional workflow.
• Figure 3: Our method takes the input of images or sketches and uses a feature extractor with conditions added via extracted wireframe information to generate the command and parameters of a 3D CAD model. A CAD kernel can be used to convert the commands and parameters to a 3D model.
• Figure 4: The visualization of the KOCAD dataset and the corresponding command sequences.
• Figure 5: More Examples of KOCAD Dataset. Objects are fabricated with different materials and captured under varying environmental conditions.