MVImgNet2.0: A Larger-scale Dataset of Multi-view Images

TL;DR

MVImgNet2.0 substantially enlarges the real-world multi-view 3D dataset landscape by expanding to 520k objects across 515 categories and delivering 360-degree captures with higher-quality masks, poses, and dense reconstructions. The authors redesign annotation pipelines (PixSfM-based poses, advanced segmentation, Neural-Angelo-based dense reconstruction) and demonstrate that MVImgNet2.0 data improves per-scene and category-agnostic 3D reconstruction across multiple baselines. Comprehensive experiments reveal that increased data scale, broader category coverage, and 360° views yield measurable gains in PSNR and Chamfer metrics, confirming MVImgNet2.0’s value as a large-scale 3D prior. The dataset and annotations are slated for public release to spur progress in 3D reconstruction, radiance-field methods, and view synthesis.

Abstract

MVImgNet is a large-scale dataset that contains multi-view images of ~220k real-world objects in 238 classes. As a counterpart of ImageNet, it introduces 3D visual signals via multi-view shooting, making a soft bridge between 2D and 3D vision. This paper constructs the MVImgNet2.0 dataset that expands MVImgNet into a total of ~520k objects and 515 categories, which derives a 3D dataset with a larger scale that is more comparable to ones in the 2D domain. In addition to the expanded dataset scale and category range, MVImgNet2.0 is of a higher quality than MVImgNet owing to four new features: (i) most shoots capture 360-degree views of the objects, which can support the learning of object reconstruction with completeness; (ii) the segmentation manner is advanced to produce foreground object masks of higher accuracy; (iii) a more powerful structure-from-motion method is adopted to derive the camera pose for each frame of a lower estimation error; (iv) higher-quality dense point clouds are reconstructed via advanced methods for objects captured in 360-degree views, which can serve for downstream applications. Extensive experiments confirm the value of the proposed MVImgNet2.0 in boosting the performance of large 3D reconstruction models. MVImgNet2.0 will be public at luyues.github.io/mvimgnet2, including multi-view images of all 520k objects, the reconstructed high-quality point clouds, and data annotation codes, hoping to inspire the broader vision community.

Figures (14)

• Figure 1: MVImgNet2.0 data visualization. Objects in MVImgNet2.0 are in a wide range. For each object, we visualize the estimated camera poses and then sample 4 views to present images (whose corresponding camera poses are highlighted in dark color). We also visualize the point cloud annotations (PCL).
• Figure 2: The data acquisition and annotation pipeline in MVImgNet2.0. One video is first collected, uploaded, and qualified by collectors and annotators, then we extract frames from the video to conduct annotation including camera pose estimation via PixSfM, then object segmentation via a detection-segmentation-tracking pipeline, and lastly dense point cloud reconstruction via Instant-Angelo. All annotations are qualified by human annotators finally to filter out failure cases. New features in the MVImgNet2.0 pipeline are highlighted in brown or red color.
• Figure 3: The sparse reconstruction results comparison between using the original approach in MVImgNet (MV1-Anno) and using our advanced approach (MV2-Anno) for camera pose estimation (cameras are visualized in purple).
• Figure 4: The comparison of the foreground object segmentation results between using the original approach in MVImgNet (MV1-Anno) and using our advanced approach (MV2-Anno).
• Figure 5: The dense point cloud reconstruction results comparison between using the original approach in MVImgNet (MV1-Anno) and using our advanced approach (MV2-Anno).