PEAR: Pixel-aligned Expressive humAn mesh Recovery

TL;DR

PEAR addresses the need for fast and accurate monocular 3D human mesh recovery by introducing EHM-s, a joint SMPLX-FLAME model augmented with a head-scale parameter, and a single ViT backbone. It employs pixel-level supervision via a differentiable renderer and a two-stage training regime to recover expressive body, hands, and facial details in real time. A part-level pseudo-label refinement strategy provides robust supervision across diverse crops, enabling strong generalization. The approach delivers substantial pose and facial expression improvements over SMPLX-based baselines and enables practical real-time avatar and VR/AR applications, achieving over 100 FPS on modern GPUs.

Abstract

Reconstructing detailed 3D human meshes from a single in-the-wild image remains a fundamental challenge in computer vision. Existing SMPLX-based methods often suffer from slow inference, produce only coarse body poses, and exhibit misalignments or unnatural artifacts in fine-grained regions such as the face and hands. These issues make current approaches difficult to apply to downstream tasks. To address these challenges, we propose PEAR-a fast and robust framework for pixel-aligned expressive human mesh recovery. PEAR explicitly tackles three major limitations of existing methods: slow inference, inaccurate localization of fine-grained human pose details, and insufficient facial expression capture. Specifically, to enable real-time SMPLX parameter inference, we depart from prior designs that rely on high resolution inputs or multi-branch architectures. Instead, we adopt a clean and unified ViT-based model capable of recovering coarse 3D human geometry. To compensate for the loss of fine-grained details caused by this simplified architecture, we introduce pixel-level supervision to optimize the geometry, significantly improving the reconstruction accuracy of fine-grained human details. To make this approach practical, we further propose a modular data annotation strategy that enriches the training data and enhances the robustness of the model. Overall, PEAR is a preprocessing-free framework that can simultaneously infer EHM-s (SMPLX and scaled-FLAME) parameters at over 100 FPS. Extensive experiments on multiple benchmark datasets demonstrate that our method achieves substantial improvements in pose estimation accuracy compared to previous SMPLX-based approaches. Project page: https://wujh2001.github.io/PEAR

Figures (17)

• Figure 1: We propose PEAR, a pixel-aligned human mesh recovery framework that surpasses prior SMPLX-based methods and is robust to diverse human body crops. It recovers more accurate facial details and body poses from a single image in under 0.01s without body-part cropping, making it well-suited for realtime downstream applications.
• Figure 2: Current methods overview. MHR is a full-body human model; however, SAM3D-body predicts only the body and hand components of MHR.
• Figure 3: Expressive human mesh recovery method comparison. Existing SMPLX-based methods mainly adopt the two architectures shown on the top. In contrast, we introduce pixel-level supervision without increasing model complexity, enabling fast inference with high-quality body and face reconstruction. This is not trivial, and details are provided below.
• Figure 4: Overview of PEAR. PEAR adopts a unified ViT backbone (includes encoder and decoder) to jointly regress SMPLX body parameters and FLAME-consistent head parameters. A scale parameter $s$ is introduced to account for head size variations, enabling robust modeling across both children and adults while maintaining efficient inference.
• Figure 5: Pixel-aligned training strategy for enhancing PEAR.