MV2Cyl: Reconstructing 3D Extrusion Cylinders from Multi-View Images

TL;DR

MV2Cyl, a novel method for reconstructing 3D from 2D multi-view images, not merely as a field or raw geometry but as a sketch-extrude CAD model that achieves the optimal reconstruction result with the best accuracy in 2D sketch and extrude parameter estimation.

Abstract

We present MV2Cyl, a novel method for reconstructing 3D from 2D multi-view images, not merely as a field or raw geometry but as a sketch-extrude CAD model. Extracting extrusion cylinders from raw 3D geometry has been extensively researched in computer vision, while the processing of 3D data through neural networks has remained a bottleneck. Since 3D scans are generally accompanied by multi-view images, leveraging 2D convolutional neural networks allows these images to be exploited as a rich source for extracting extrusion cylinder information. However, we observe that extracting only the surface information of the extrudes and utilizing it results in suboptimal outcomes due to the challenges in the occlusion and surface segmentation. By synergizing with the extracted base curve information, we achieve the optimal reconstruction result with the best accuracy in 2D sketch and extrude parameter estimation. Our experiments, comparing our method with previous work that takes a raw 3D point cloud as input, demonstrate the effectiveness of our approach by taking advantage of multi-view images. Our project page can be found at http://mv2cyl.github.io .

Figures (15)

• Figure 1: Full Pipeline of MV2Cyl.
• Figure 2: Example of segmentation prediction. From left to right: input rendered image, surface instance segmentation, surface start-end-barrel segmentation, curve instance segmentation, and curve start-end segmentation.
• Figure 3: Overview of the learned surface and curve fields. (Left-to-Right) Density field of surface, instance semantic field of surface, start-end semantic field of surface, density field of curve, instance semantic field of curve, and start-end semantic field of curve.
• Figure 4: Converting 3D reconstructed geometry and semantics into CAD parameters.
• Figure 5: Qualitative comparisons with the baselines. Each instance is identified by a different color. MV2Cyl produces high-quality geometry and even outperforms Point2Cyl uy2022point2cyl that directly consumes 3D point clouds. Furthermore, the comparison against a naive baseline that pipelines NeuS2 neus2, a multi-view surface reconstruction technique, to Point2Cyl demonstrates the importance of edge information when inferring 3D structures.