Neural-ABC: Neural Parametric Models for Articulated Body with Clothes

TL;DR

Neural-ABC addresses the challenge of representing clothed human bodies with diverse garments and poses by introducing a neural implicit parametric framework with four disentangled latent spaces: identity, clothing, shape, and pose. It unifies body and clothing into a cascade of signed and unsigned distance fields and applies pose through linear blend skinning, enabling topology-free clothing and accurate fitting to scans, depth maps, and images. The method is trained with a comprehensive data pipeline that includes real scans, CLOTH3D, and a newly created DressUp synthetic dataset to provide decoupled information necessary for independent control of each latent space. The results show superior clothing representation, flexible editing of identity/clothing/shape/pose, and practical applications such as clothing transfer and animation, with DressUp supporting decoupled learning. This work lays groundwork for flexible digital humans by combining neural implicit geometry, decoupled latent spaces, and pose-aware clothing deformation.

Abstract

In this paper, we introduce Neural-ABC, a novel parametric model based on neural implicit functions that can represent clothed human bodies with disentangled latent spaces for identity, clothing, shape, and pose. Traditional mesh-based representations struggle to represent articulated bodies with clothes due to the diversity of human body shapes and clothing styles, as well as the complexity of poses. Our proposed model provides a unified framework for parametric modeling, which can represent the identity, clothing, shape and pose of the clothed human body. Our proposed approach utilizes the power of neural implicit functions as the underlying representation and integrates well-designed structures to meet the necessary requirements. Specifically, we represent the underlying body as a signed distance function and clothing as an unsigned distance function, and they can be uniformly represented as unsigned distance fields. Different types of clothing do not require predefined topological structures or classifications, and can follow changes in the underlying body to fit the body. Additionally, we construct poses using a controllable articulated structure. The model is trained on both open and newly constructed datasets, and our decoupling strategy is carefully designed to ensure optimal performance. Our model excels at disentangling clothing and identity in different shape and poses while preserving the style of the clothing. We demonstrate that Neural-ABC fits new observations of different types of clothing. Compared to other state-of-the-art parametric models, Neural-ABC demonstrates powerful advantages in the reconstruction of clothed human bodies, as evidenced by fitting raw scans, depth maps and images. We show that the attributes of the fitted results can be further edited by adjusting their identities, clothing, shape and pose codes.

Figures (25)

• Figure 1: Neural-ABC is a neural implicit parametric model with latent spaces of human identity, clothing, shape and pose. It can generate various human identities and different clothes. The clothed human body can deform into different body shapes and poses.
• Figure 2: Overview of our Neural-ABC pipeline. Given query points and latent codes, our model can generate the corresponding human body geometry by three modules.
• Figure 3: In the pose space, we use three latent spaces to encode query points in canonical pose. First, calculate the pose dependent offset for all query points. Then transfer them to the posed space by neural LBS.
• Figure 4: We show the objects of different body types wearing the same clothes in the DressUp dataset.
• Figure 5: The real scan (a) is first registered as SMPL+D (b), and then the SMPL+D in the canonical pose (c) is obtained. Using manually annotated segmentation information (d) can obtain separate clothing (e). The SMPL in canonical pose (f) as a body agent for the clothed human body (g) for training. (h) is the clothing in posed pace. The geometry inside the rectangular box is used to train our model.