Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MVHumanNet++: A Large-scale Dataset of Multi-view Daily Dressing Human Captures with Richer Annotations for 3D Human Digitization

Chenghong Li, Hongjie Liao, Yihao Zhi, Xihe Yang, Zhengwentai Sun, Jiahao Chang, Shuguang Cui, Xiaoguang Han

TL;DR

MVHumanNet++ tackles the scarcity of large-scale 3D human data by delivering a comprehensive, multi-view dataset with 4,500 identities, 9,000 outfits, 60,000 motion sequences, and 645 million frames, complemented by masks, camera parameters, 2D/3D keypoints, SMPL/SMPLX parameters, text descriptions, and newly processed normal and depth maps. The authors validate the dataset through pilot studies spanning view-consistent action recognition, NeRF/3DGS reconstruction, and text-driven generation, demonstrating that scale and rich annotations improve generalization and reconstruction fidelity. By releasing this resource, they aim to accelerate 3D human digitization research and enable scalable, real-world applications across 2D/3D understanding, generation, and avatar creation. The work positions MVHumanNet++ as a foundational benchmark for scalable, human-centric tasks in the 3D vision community.

Abstract

In this era, the success of large language models and text-to-image models can be attributed to the driving force of large-scale datasets. However, in the realm of 3D vision, while significant progress has been achieved in object-centric tasks through large-scale datasets like Objaverse and MVImgNet, human-centric tasks have seen limited advancement, largely due to the absence of a comparable large-scale human dataset. To bridge this gap, we present MVHumanNet++, a dataset that comprises multi-view human action sequences of 4,500 human identities. The primary focus of our work is on collecting human data that features a large number of diverse identities and everyday clothing using multi-view human capture systems, which facilitates easily scalable data collection. Our dataset contains 9,000 daily outfits, 60,000 motion sequences and 645 million frames with extensive annotations, including human masks, camera parameters, 2D and 3D keypoints, SMPL/SMPLX parameters, and corresponding textual descriptions. Additionally, the proposed MVHumanNet++ dataset is enhanced with newly processed normal maps and depth maps, significantly expanding its applicability and utility for advanced human-centric research. To explore the potential of our proposed MVHumanNet++ dataset in various 2D and 3D visual tasks, we conducted several pilot studies to demonstrate the performance improvements and effective applications enabled by the scale provided by MVHumanNet++. As the current largest-scale 3D human dataset, we hope that the release of MVHumanNet++ dataset with annotations will foster further innovations in the domain of 3D human-centric tasks at scale. MVHumanNet++ is publicly available at https://kevinlee09.github.io/research/MVHumanNet++/.

MVHumanNet++: A Large-scale Dataset of Multi-view Daily Dressing Human Captures with Richer Annotations for 3D Human Digitization

TL;DR

MVHumanNet++ tackles the scarcity of large-scale 3D human data by delivering a comprehensive, multi-view dataset with 4,500 identities, 9,000 outfits, 60,000 motion sequences, and 645 million frames, complemented by masks, camera parameters, 2D/3D keypoints, SMPL/SMPLX parameters, text descriptions, and newly processed normal and depth maps. The authors validate the dataset through pilot studies spanning view-consistent action recognition, NeRF/3DGS reconstruction, and text-driven generation, demonstrating that scale and rich annotations improve generalization and reconstruction fidelity. By releasing this resource, they aim to accelerate 3D human digitization research and enable scalable, real-world applications across 2D/3D understanding, generation, and avatar creation. The work positions MVHumanNet++ as a foundational benchmark for scalable, human-centric tasks in the 3D vision community.

Abstract

In this era, the success of large language models and text-to-image models can be attributed to the driving force of large-scale datasets. However, in the realm of 3D vision, while significant progress has been achieved in object-centric tasks through large-scale datasets like Objaverse and MVImgNet, human-centric tasks have seen limited advancement, largely due to the absence of a comparable large-scale human dataset. To bridge this gap, we present MVHumanNet++, a dataset that comprises multi-view human action sequences of 4,500 human identities. The primary focus of our work is on collecting human data that features a large number of diverse identities and everyday clothing using multi-view human capture systems, which facilitates easily scalable data collection. Our dataset contains 9,000 daily outfits, 60,000 motion sequences and 645 million frames with extensive annotations, including human masks, camera parameters, 2D and 3D keypoints, SMPL/SMPLX parameters, and corresponding textual descriptions. Additionally, the proposed MVHumanNet++ dataset is enhanced with newly processed normal maps and depth maps, significantly expanding its applicability and utility for advanced human-centric research. To explore the potential of our proposed MVHumanNet++ dataset in various 2D and 3D visual tasks, we conducted several pilot studies to demonstrate the performance improvements and effective applications enabled by the scale provided by MVHumanNet++. As the current largest-scale 3D human dataset, we hope that the release of MVHumanNet++ dataset with annotations will foster further innovations in the domain of 3D human-centric tasks at scale. MVHumanNet++ is publicly available at https://kevinlee09.github.io/research/MVHumanNet++/.
Paper Structure (15 sections, 20 figures, 10 tables, 1 algorithm)

This paper contains 15 sections, 20 figures, 10 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. MVHumanNet++
  4. Multi-view Synchronized Capture System
  5. Data Capture and Statistics
  6. Data Annotation
  7. Experiments
  8. View-consistent Action Recognition
  9. NeRF Reconstruction for Human
  10. Per-Subject 3DGS Reconstruction for Human
  11. Generalizable 3DGS Reconstruction for Human
  12. Text-driven Image Generation
  13. Human Generative Model
  14. Reconstruction from Unconstraint Human Images
  15. Conclusions

Figures (20)

  • Figure 1: We introduce MVHumanNet++, a large-scale dataset of multi-view human images with unprecedented scale in human subjects, daily outfits, motion sequences and frames. Top left and right: Examples of multi-view poses featuring different human identities with various daily dressing in our dataset. Top middle: Our multi-view capture system includes 48 cameras of 12MP resolution. Bottom: Comprehensive visualization of all 9000 outfits in our MVHumanNet++.
  • Figure 2: The distribution of performers' attributes. The gender, age, weight, and height of performers are recorded and carefully controlled. The statistical analysis of these attributes reflects a diverse range among the performers involved in MVHumanNet++.
  • Figure 3: The garment type and color distribution of outfits of performers. Diverse colors and types of dressing are required for each invited performer. The statistical results show the wide coverage of daily clothes.
  • Figure 4: Data annotation pipeline. The manual and automatic annotation pipeline for action localization, text description, masks, 2D/3D keypoints, parametric models, normal maps and depth maps.
  • Figure 5: A text description example. The description contains various information, such as age, height, garment and hairstyle.
  • ...and 15 more figures