EfficientHuman: Efficient Training and Reconstruction of Moving Human using Articulated 2D Gaussian

TL;DR

EfficientHuman tackles rapid dynamic 3D human reconstruction from monocular video by replacing 3D Gaussian ellipsoids with Articulated 2D Gaussian surfels encoded in a canonical space and mapped to pose space via Linear Blend Skinning. It introduces a pose calibration module and an LBS optimization module to align the Gaussians with SMPL-based priors, enabling fast fitting and high rendering quality. On the ZJU-MoCap dataset, it achieves training in under a minute with 1.2K iterations and reduces redundant Gaussians by about 17%, while preserving PSNR/SSIM/LPIPS quality comparable to state-of-the-art. Ablation studies validate the importance of LBS optimization, pose calibration, and the mask loss for accurate reconstruction. The approach offers a practical, time-efficient alternative for dynamic human capture and has potential for downstream mesh extraction and texture mapping.

Abstract

3D Gaussian Splatting (3DGS) has been recognized as a pioneering technique in scene reconstruction and novel view synthesis. Recent work on reconstructing the 3D human body using 3DGS attempts to leverage prior information on human pose to enhance rendering quality and improve training speed. However, it struggles to effectively fit dynamic surface planes due to multi-view inconsistency and redundant Gaussians. This inconsistency arises because Gaussian ellipsoids cannot accurately represent the surfaces of dynamic objects, which hinders the rapid reconstruction of the dynamic human body. Meanwhile, the prevalence of redundant Gaussians means that the training time of these works is still not ideal for quickly fitting a dynamic human body. To address these, we propose EfficientHuman, a model that quickly accomplishes the dynamic reconstruction of the human body using Articulated 2D Gaussian while ensuring high rendering quality. The key innovation involves encoding Gaussian splats as Articulated 2D Gaussian surfels in canonical space and then transforming them to pose space via Linear Blend Skinning (LBS) to achieve efficient pose transformations. Unlike 3D Gaussians, Articulated 2D Gaussian surfels can quickly conform to the dynamic human body while ensuring view-consistent geometries. Additionally, we introduce a pose calibration module and an LBS optimization module to achieve precise fitting of dynamic human poses, enhancing the model's performance. Extensive experiments on the ZJU-MoCap dataset demonstrate that EfficientHuman achieves rapid 3D dynamic human reconstruction in less than a minute on average, which is 20 seconds faster than the current state-of-the-art method, while also reducing the number of redundant Gaussians.

Figures (3)

• Figure 1: EfficientHuman receives the monocular videos to model a 3D human with currently the shortest training time and ensuring high-quality rendering. The left images provide a comparative analysis of the training efficiency and image quality between the current state-of-the-art and our proposed EfficientHuman method, our work not only guarantees the integrity of clothing edges but also achieves this with a significantly reduced training time. The right comparison plot demonstrates the Peak Signal-to-Noise Ratio (PSNR) and Training Time, highlighting the superior performance of EfficientHuman in ensuring PSNR within a significantly reduced training duration.
• Figure 2: EfficientHuman method overview. Our pipeline is initiated with the ingestion of sequential video frames, followed by the initialization of Gaussians within a predefined canonical space. These Gaussians are subsequently encoded and articulated into a coherent set of 2D Gaussian surfels with human prior, embodying the subject in an anatomical T-pose. Notably, the 2D Gaussian surfels in our method feature negligible gaps, in contrast to the more substantial separations found between Gaussian ellipsoids. The transformation and control of these surfels are directed by the optimized parameters $\theta_T$ and $p_c^e$, which are optimized through modules of pose calibration and Linear Blend Skinning (LBS) optimization, respectively. Finally, the 2D Gaussian surfels are mapped into the pose space of the target camera view using LBS.
• Figure 3: Qualitative comparison of our approach and the baseline method on ZJU-MoCap, highlighting six distinct actions. The rendering quality of our method is comparable with the baseline method, and Figure 3 also illustrates the improvement of our reconstruction method over the baseline, particularly in the red-boxed area of row three and columns one to eight. This is evident in the preservation of fine details, such as the clothing button and edges, as well as the improved recovery and rendering of human arm contours. Zoom in for the best view.