Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Investigating Anthropometric Fidelity in SAM 3D Body

Aizierjiang Aiersilan, Ruting Cheng, James Hahn

TL;DR

This work analyzes why SAM 3D Body, a robust single-image human mesh method based on the Momentum Human Rig, under-reconstructs anthropometric deviations such as pregnancy, scoliosis, and geriatric atrophy. It identifies three interacting causes—a parametric bottleneck in MHR, semantic collapse from DINOv3 conditioning, and annotation/alignment practices that bias toward a mean morphology—formalizing these with distributions and priors and illustrating the resulting perception–distortion trade-off. The authors propose concrete pathways to bridge the gap to medical utility, including implicit–explicit hybrid representations, medical-domain expert alignment, and parametric injection of extended body models to decouple global type from local pathology. While not diagnostic-ready, the model's robust reconstruction baseline can serve medical simulation, base meshes for deformation, and focused fine-tuning on domain-specific datasets, enabling a principled route toward medical-in-the-loop 3D human reconstruction.

Abstract

The recent release of SAM 3D Body \cite{sam3dbody2025} marks a significant milestone in human mesh recovery, demonstrating state-of-the-art performance in producing clean, topologically coherent meshes from single images. By leveraging the novel Momentum Human Rig (MHR), it achieves remarkable robustness to occlusion and diverse poses. However, our evaluation reveals a specific and consistent limitation: the model struggles to reconstruct detailed anthropometric deviations, especially on populations with special body shape alters such as geriatric muscle atrophy, scoliosis, or pregnancy, even when these features are prominent in the input image. In this paper, we investigate this phenomenon not as a failure of the model's capacity, but as a byproduct of the \textit{perception-distortion trade-off}. We posit that the architectural reliance on the low-dimensional parametric MHR representation, combined with semantic-invariant conditioning (DINOv3) and annotation-based alignment, creates a \enquote{regression to the mean} effect. We analyze these mechanisms to understand why individual biological details are smoothed out and propose specific, constructive pathways for future work to extend the impressive baseline performance of SAM 3D Body into the medical domain.

Investigating Anthropometric Fidelity in SAM 3D Body

TL;DR

This work analyzes why SAM 3D Body, a robust single-image human mesh method based on the Momentum Human Rig, under-reconstructs anthropometric deviations such as pregnancy, scoliosis, and geriatric atrophy. It identifies three interacting causes—a parametric bottleneck in MHR, semantic collapse from DINOv3 conditioning, and annotation/alignment practices that bias toward a mean morphology—formalizing these with distributions and priors and illustrating the resulting perception–distortion trade-off. The authors propose concrete pathways to bridge the gap to medical utility, including implicit–explicit hybrid representations, medical-domain expert alignment, and parametric injection of extended body models to decouple global type from local pathology. While not diagnostic-ready, the model's robust reconstruction baseline can serve medical simulation, base meshes for deformation, and focused fine-tuning on domain-specific datasets, enabling a principled route toward medical-in-the-loop 3D human reconstruction.

Abstract

The recent release of SAM 3D Body \cite{sam3dbody2025} marks a significant milestone in human mesh recovery, demonstrating state-of-the-art performance in producing clean, topologically coherent meshes from single images. By leveraging the novel Momentum Human Rig (MHR), it achieves remarkable robustness to occlusion and diverse poses. However, our evaluation reveals a specific and consistent limitation: the model struggles to reconstruct detailed anthropometric deviations, especially on populations with special body shape alters such as geriatric muscle atrophy, scoliosis, or pregnancy, even when these features are prominent in the input image. In this paper, we investigate this phenomenon not as a failure of the model's capacity, but as a byproduct of the \textit{perception-distortion trade-off}. We posit that the architectural reliance on the low-dimensional parametric MHR representation, combined with semantic-invariant conditioning (DINOv3) and annotation-based alignment, creates a \enquote{regression to the mean} effect. We analyze these mechanisms to understand why individual biological details are smoothed out and propose specific, constructive pathways for future work to extend the impressive baseline performance of SAM 3D Body into the medical domain.
Paper Structure (20 sections, 12 equations, 2 figures, 1 table)

This paper contains 20 sections, 12 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Parametric Human Models and Phenotypic Diversity
  4. High-Fidelity and Implicit Reconstruction
  5. Biomechanical Fidelity
  6. The Architectural Smoothing of Biology
  7. The Parametric Bottleneck of MHR
  8. Semantic Invariance in DINOv3 Conditioning
  9. The Alignment Tax: Why "Better" Means "Generic"
  10. Dataset Bias in SAM 3D Body Dataset
  11. 1) Annotation Bias and Idealized Forms.
  12. 2) Missing High-Frequency Biological Detail.
  13. The Annotation Alignment Trap
  14. Pathway to Medical Utility
  15. Simulation vs. Diagnosis
  16. ...and 5 more sections

Figures (2)

  • Figure 1: The limitations of SAM 3D Body when generating 3D figures for pregnant women (left, under license from Pressmaster - stock.adobe.com), people with scoliosis scoliosisclassification_dataset (middle left), people with limb loss (middle right, under license from Kawee - stock.adobe.com), and obese people (right, under license from DenisProduction.com - stock.adobe.com). Missing biological signature of each population is marked by a red box.
  • Figure 2: Comparison between body shapes obtained by commercial scanner and SAM 3D body on both pregnant woman and non-pregnant woman. The 3D figures generated by SAM 3D body are projected from front and side on the right of the scanner captured ones.