PI-HMR: Towards Robust In-bed Temporal Human Shape Reconstruction with Contact Pressure Sensing

TL;DR

The paper tackles privacy-preserving in-bed 3D human shape reconstruction by combining a label-generation pipeline (SMPLify-IB) with a pressure-based temporal estimator (PI-HMR). PI-HMR directly regresses SMPL meshes from pressure sequences, leveraging a multi-scale feature fusion framework, spatial priors, cross-modal knowledge distillation, and test-time optimization with a VQ-VAE motion prior. To address label quality and generalization, SMPLify-IB generates high-quality SMPL pseudo-ground-truths for TIP using gravity constraints and a lightweight self-penetration penalty, significantly reducing depth-ambiguity artifacts. The results show that PI-HMR surpasses state-of-the-art pressure- and vision-based methods (e.g., MPJPE improvements of around 17 mm over PI-Mesh and competitive gains over HMR-based models), while SMPLify-IB provides reliable annotations and faster, accurate detection for in-bed poses, together enabling robust, privacy-preserving in-bed motion analysis with practical clinical impact.

Abstract

Long-term in-bed monitoring benefits automatic and real-time health management within healthcare, and the advancement of human shape reconstruction technologies further enhances the representation and visualization of users' activity patterns. However, existing technologies are primarily based on visual cues, facing serious challenges in non-light-of-sight and privacy-sensitive in-bed scenes. Pressure-sensing bedsheets offer a promising solution for real-time motion reconstruction. Yet, limited exploration in model designs and data have hindered its further development. To tackle these issues, we propose a general framework that bridges gaps in data annotation and model design. Firstly, we introduce SMPLify-IB, an optimization method that overcomes the depth ambiguity issue in top-view scenarios through gravity constraints, enabling generating high-quality 3D human shape annotations for in-bed datasets. Then we present PI-HMR, a temporal-based human shape estimator to regress meshes from pressure sequences. By integrating multi-scale feature fusion with high-pressure distribution and spatial position priors, PI-HMR outperforms SOTA methods with 17.01mm Mean-Per-Joint-Error decrease. This work provides a whole

Figures (18)

• Figure 1: We present a general framework for in-bed HPS tasks, containing a monocular optimization strategy to generate high-quality SMPL annotations in in-bed scenarios, SMPLify-IB; and a HPS network to predict in-bed motions from pressure sequence, PI-HMR.
• Figure 2: A glimpse of TIP dataset, with p-GTs from TIP and our SMPLify-IB. we highlight its drawbacks with red ellipses and our refinements in yellow ones.
• Figure 3: (a) and (b): demos of our detection algorithm. $S$ is the segment, $C$ is its segment center. $v_i$ are vertices that need to be checked for penetration with $S$, and $v_j$ are the vertices from $S$ that are closest to $v_i$, respectively. When $\overrightarrow{v_iv_j} \cdot \overrightarrow{v_iC} < 0$, $v_i$ is in penetration, and vice versa. (c) is our segment.
• Figure 4: An overview of PI-HMR. PI-HMR outputs the midframe's SMPL predictions of the whole sequence.
• Figure 5: Framework of our multi-scale feature fusion module.