Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AirNeRF: 3D Reconstruction of Human with Drone and NeRF for Future Communication Systems

Alexey Kotcov, Maria Dronova, Vladislav Cheremnykh, Sausar Karaf, Dzmitry Tsetserukou

TL;DR

AirNeRF tackles the challenge of fast, autonomous 3D human avatar reconstruction for VR and telepresence by combining drone-based data capture, NeRF-based view synthesis, semantic segmentation, and automatic rigging to produce lightweight, rigged avatars. The pipeline leverages Nerfacto for efficient NeRF training, achieving about 6 minutes of training on 300 images and generating FBX avatars around 3 MB with 22 joints and 36k–51k triangles, suitable for modern AR/VR engines. The approach demonstrates data-efficient, photogrammetry-free 3D human reconstruction in unconstrained settings and points toward future enhancements with Gaussian Splatting for even richer geometry representations. This work enables accessible, high-fidelity digital humans for VR/AR, Metaverse, and industrial applications such as BIM and telepresence.

Abstract

In the rapidly evolving landscape of digital content creation, the demand for fast, convenient, and autonomous methods of crafting detailed 3D reconstructions of humans has grown significantly. Addressing this pressing need, our AirNeRF system presents an innovative pathway to the creation of a realistic 3D human avatar. Our approach leverages Neural Radiance Fields (NeRF) with an automated drone-based video capturing method. The acquired data provides a swift and precise way to create high-quality human body reconstructions following several stages of our system. The rigged mesh derived from our system proves to be an excellent foundation for free-view synthesis of dynamic humans, particularly well-suited for the immersive experiences within gaming and virtual reality.

AirNeRF: 3D Reconstruction of Human with Drone and NeRF for Future Communication Systems

TL;DR

AirNeRF tackles the challenge of fast, autonomous 3D human avatar reconstruction for VR and telepresence by combining drone-based data capture, NeRF-based view synthesis, semantic segmentation, and automatic rigging to produce lightweight, rigged avatars. The pipeline leverages Nerfacto for efficient NeRF training, achieving about 6 minutes of training on 300 images and generating FBX avatars around 3 MB with 22 joints and 36k–51k triangles, suitable for modern AR/VR engines. The approach demonstrates data-efficient, photogrammetry-free 3D human reconstruction in unconstrained settings and points toward future enhancements with Gaussian Splatting for even richer geometry representations. This work enables accessible, high-fidelity digital humans for VR/AR, Metaverse, and industrial applications such as BIM and telepresence.

Abstract

In the rapidly evolving landscape of digital content creation, the demand for fast, convenient, and autonomous methods of crafting detailed 3D reconstructions of humans has grown significantly. Addressing this pressing need, our AirNeRF system presents an innovative pathway to the creation of a realistic 3D human avatar. Our approach leverages Neural Radiance Fields (NeRF) with an automated drone-based video capturing method. The acquired data provides a swift and precise way to create high-quality human body reconstructions following several stages of our system. The rigged mesh derived from our system proves to be an excellent foundation for free-view synthesis of dynamic humans, particularly well-suited for the immersive experiences within gaming and virtual reality.
Paper Structure (14 sections, 3 equations, 7 figures, 3 tables)

This paper contains 14 sections, 3 equations, 7 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Quality Comparison of the 3D Reconstruction Approaches
  4. Static model
  5. Human model
  6. System Pipeline
  7. Drone configuration
  8. NeRF model configuration
  9. System Evaluation
  10. Dataset collection
  11. Human body segmentation
  12. NeRF training
  13. Mesh rigging
  14. Conclusion and Future Work

Figures (7)

  • Figure 1: Layout of AirNeRF system for 3D human reconstruction.
  • Figure 2: Comparison of 3D reconstruction methods: a) Photogrammetry using scanner. b) Photogrammetry from the set of images. c) Nerfacto model. d) Instant-NGP model. e) Gaussian Splatting render, on a static reduced-size human figure.
  • Figure 3: Hardware architecture of the AirNeRF system.
  • Figure 4: Data capturing process by the drone following the helical trajectory.
  • Figure 5: Segmentation pipeline for extracting human outline from the collected data.
  • ...and 2 more figures