CHRIS: Clothed Human Reconstruction with Side View Consistency

TL;DR

CHRIS tackles monocular clothed-human reconstruction by explicitly regularizing side-view geometry. It introduces a Side-View Normal Discriminator to enforce global side-view plausibility and a Multi-to-One Gradient Computation to ensure local surface consistency, both integrated into an implicit SDF-based framework conditioned on SMPL-X. The method yields state-of-the-art Chamfer, P2S, and normal metrics on THuman2.0, CAPE, and in-the-wild images, and produces animatable 3D avatars. This work advances single-view clothed-human reconstruction by better exploiting side-view cues for realistic topology and wrinkle details.

Abstract

Creating a realistic clothed human from a single-view RGB image is crucial for applications like mixed reality and filmmaking. Despite some progress in recent years, mainstream methods often fail to fully utilize side-view information, as the input single-view image contains front-view information only. This leads to globally unrealistic topology and local surface inconsistency in side views. To address these, we introduce Clothed Human Reconstruction with Side View Consistency, namely CHRIS, which consists of 1) A Side-View Normal Discriminator that enhances global visual reasonability by distinguishing the generated side-view normals from the ground truth ones; 2) A Multi-to-One Gradient Computation (M2O) that ensures local surface consistency. M2O calculates the gradient of a sampling point by integrating the gradients of the nearby points, effectively acting as a smooth operation. Experimental results demonstrate that CHRIS achieves state-of-the-art performance on public benchmarks and outperforms the prior work.

Figures (5)

• Figure 1: Comparisons with recent advances in in-the-wild images, our CHRIS achieves more accurate and visually reasonable reconstruction of challenging poses and diverse clothes. Zoom in for more details.
• Figure 2: Overview of our CHRIS. Conditioned on a single-view image $\mathcal{I}$ and the corresponding SMPL-X $\mathcal{M}$, we sample 3D points $\mathbf{P}$, obtaining their features regarding the geometry of the 3D clothed human. Then, we obtain the SDF $\hat{\mathcal{S}}$ and the corresponding four-view normal maps $\hat{\mathcal{N}}^c$ w.r.t. the clothed human from $\mathcal{I}$. To improve the global side-view geometry, we introduce a Side-View Normal Discriminator $\mathcal{D}$ that distinguishes between $\hat{\mathcal{N}}_{side}^{c}= \{ \hat{N}_{left}^{c}, \hat{{N}}_{right}^{c}\}$ and ground truth $\mathcal{N}^c$. Moreover, to enhance local surface consistency, we employ a Multi-to-one Gradient Computation combined with a coarse-to-fine strategy. By integrating the gradients of nearby points around $p$, we smooth out local irregularities, leading to consistent surface reconstruction $\mathcal{M}_{occ}$. From subfigure (a), we sample two additional points along each axis (x, y, and z) of the canonical 3D coordinate system around a single point $\mathbf{p}_i$, with a step size $\epsilon$. The value of $\epsilon$ decreases as training proceeds. Additionally, we can substitute the hands with SMPL-X models and map textures to mesh for enhanced visuals.
• Figure 3: Visualization results of 3D clothed avatar reconstruction with our CHRIS from in-the-wild images, which present various clothing and challenging poses. We show the front (blue) and side (red) views. Zoom in for more details.
• Figure 4: Reconstruction results on CAPE dataset with or without our Side-View Normal Discriminator. All the images are side normal maps relative to the input image. Side-View Normal Discriminator can greatly enhance the geometry topology in the side view. The non-human parts are removed and the contours are more reasonable.
• Figure 5: Reconstruction results on CAPE dataset with or without our Multi-to-one Gradient Computation. The results show that our Multi-to-one Gradient Computation can effectively eliminate the jagged surfaces and produce more consistent and more realistic geometry.