Recovering 3D Human Mesh from Monocular Images: A Survey
Yating Tian, Hongwen Zhang, Yebin Liu, Limin Wang
TL;DR
This survey systematically catalogs monocular 3D human mesh recovery, contrasting optimization-based fitting with regression-based learning and detailing their respective data terms, priors, representations, and architectures. It surveys body-only and unified full-body models (including hands and face), discusses temporal and multi-person extensions, and analyzes physical plausibility through camera models, contact constraints, and priors. A comprehensive review of datasets, evaluation metrics, and benchmark results is provided, along with insights into limitations and practical impacts such as AR/VR, virtual try-on, and realistic avatar creation. The paper also outlines open challenges under occlusions, video stability, scene constraints, and clothing detail, proposing future directions toward unsupervised learning, group reconstruction, and more expressive whole-body models.
Abstract
Estimating human pose and shape from monocular images is a long-standing problem in computer vision. Since the release of statistical body models, 3D human mesh recovery has been drawing broader attention. With the same goal of obtaining well-aligned and physically plausible mesh results, two paradigms have been developed to overcome challenges in the 2D-to-3D lifting process: i) an optimization-based paradigm, where different data terms and regularization terms are exploited as optimization objectives; and ii) a regression-based paradigm, where deep learning techniques are embraced to solve the problem in an end-to-end fashion. Meanwhile, continuous efforts are devoted to improving the quality of 3D mesh labels for a wide range of datasets. Though remarkable progress has been achieved in the past decade, the task is still challenging due to flexible body motions, diverse appearances, complex environments, and insufficient in-the-wild annotations. To the best of our knowledge, this is the first survey that focuses on the task of monocular 3D human mesh recovery. We start with the introduction of body models and then elaborate recovery frameworks and training objectives by providing in-depth analyses of their strengths and weaknesses. We also summarize datasets, evaluation metrics, and benchmark results. Open issues and future directions are discussed in the end, hoping to motivate researchers and facilitate their research in this area. A regularly updated project page can be found at https://github.com/tinatiansjz/hmr-survey.