BodyMAP -- Jointly Predicting Body Mesh and 3D Applied Pressure Map for People in Bed

TL;DR

BodyMAP addresses the practical problem of predicting both a 3D body mesh and a 3D applied pressure map for people in bed to aid pressure-ulcer prevention. It introduces a unified multimodal architecture that fuses depth and 2D pressure inputs to jointly predict the SMPL-based body mesh $\hat{\Psi}$ and a per-vertex pressure map $\hat{P}$, using a Feature Indexing Module to transfer image features to mesh vertices and a PointNet-based decoder for per-vertex pressures, with $\hat{P} = \hat{p} \odot \hat{c}$. The paper also proposes BodyMAP-WS, a weakly supervised variant that learns 3D pressure maps without direct 3D labels by projecting the predicted 3D pressure to 2D and aligning with the observed pressure image, enabling learning from unlabeled data. Across synthetic BodyPressureSD and real SLP datasets, BodyMAP-PointNet achieves about 25% improvements over prior state-of-the-art methods on both body mesh and 3D pressure tasks, while BodyMAP-WS delivers an 8.8% improvement in 3D pressure prediction without ground-truth 3D pressure maps. The work demonstrates substantial gains in accuracy and real-time inference, offering interpretable visualizations of pressure distribution for caregivers and enabling scalable, privacy-preserving in-bed monitoring in clinical settings.

Abstract

Accurately predicting the 3D human posture and the pressure exerted on the body for people resting in bed, visualized as a body mesh (3D pose & shape) with a 3D pressure map, holds significant promise for healthcare applications, particularly, in the prevention of pressure ulcers. Current methods focus on singular facets of the problem -- predicting only 2D/3D poses, generating 2D pressure images, predicting pressure only for certain body regions instead of the full body, or forming indirect approximations to the 3D pressure map. In contrast, we introduce BodyMAP, which jointly predicts the human body mesh and 3D applied pressure map across the entire human body. Our network leverages multiple visual modalities, incorporating both a depth image of a person in bed and its corresponding 2D pressure image acquired from a pressure-sensing mattress. The 3D pressure map is represented as a pressure value at each mesh vertex and thus allows for precise localization of high-pressure regions on the body. Additionally, we present BodyMAP-WS, a new formulation of pressure prediction in which we implicitly learn pressure in 3D by aligning sensed 2D pressure images with a differentiable 2D projection of the predicted 3D pressure maps. In evaluations with real-world human data, our method outperforms the current state-of-the-art technique by 25% on both body mesh and 3D applied pressure map prediction tasks for people in bed.

Figures (11)

• Figure 1: BodyMAP leverages a depth and pressure image of a person in bed covered by a blanket, to jointly predict the body mesh and a 3D pressure map of pressure distributed along the human body.
• Figure 2: Distinct postures can have similar 2D pressure images. The insets of the 3D pressure map show pressure being applied to different areas demonstrating its use in localizing pressure that is applied on the human body.
• Figure 3: BodyMAP jointly predicts body mesh and 3D applied pressure map for an individual in bed. (a) Model architecture that encodes depth image d and 2D pressure image p to predict SMPL loper2023smpl parameters $\hat{\Psi}$, used to reconstruct the SMPL mesh $\hat{M}$. The Feature Indexing Module (FIM) accumulates features for the mesh vertices from the input images and ResNet features. PointNet predicts the per-vertex pressure $\hat{p}$ & per-vertex binary contact $\hat{c}$ along the human body using the mesh features as input. Finally, the 3D pressure map $\hat{P}$ is calculated as the product of the per-vertex pressure value and binary contact value. (b) FIM overlays the predicted mesh over the ResNet feature maps ($\hat{l}$) and input images by mapping mesh vertex locations to pixel positions and then assigns features to each vertex (V). These are fused along with the mesh vertex locations and used for 3D pressure map prediction. (c) FIM's 'overlay and assign' step visualization.
• Figure 4: Results of inferring body mesh and 3D applied pressure map for examples from the SLP liu2019seeingliu2022simultaneously test set.
• Figure 5: Depth and pressure image complement each other. Image of overlaid modalities (third column) depicts the enhanced context available to the model.