Multimodal Active Measurement for Human Mesh Recovery in Close Proximity

TL;DR

This work proposes an active measurement and sensor fusion framework of the equipped cameras with touch and ranging sensors such as 2D LiDAR that outperformed previous methods on the standard occlusion benchmark with simulated active measurement and reliably estimated human poses using a real robot.

Abstract

For physical human-robot interactions (pHRI), a robot needs to estimate the accurate body pose of a target person. However, in these pHRI scenarios, the robot cannot fully observe the target person's body with equipped cameras because the target person must be close to the robot for physical interaction. This close distance leads to severe truncation and occlusions and thus results in poor accuracy of human pose estimation. For better accuracy in this challenging environment, we propose an active measurement and sensor fusion framework of the equipped cameras with touch and ranging sensors such as 2D LiDAR. Touch and ranging sensor measurements are sparse but reliable and informative cues for localizing human body parts. In our active measurement process, camera viewpoints and sensor placements are dynamically optimized to measure body parts with higher estimation uncertainty, which is closely related to truncation or occlusion. In our sensor fusion process, assuming that the measurements of touch and ranging sensors are more reliable than the camera-based estimations, we fuse the sensor measurements to the camera-based estimated pose by aligning the estimated pose towards the measured points. Our proposed method outperformed previous methods on the standard occlusion benchmark with simulated active measurement. Furthermore, our method reliably estimated human poses using a real robot, even with practical constraints such as occlusion by blankets.

Figures (6)

• Figure 1: Problem of previous human pose estimation methods in pHRI scenarios and our proposed method.
• Figure 2: Overview of our proposed method.
• Figure 3: Evaluated poses in two pose groups: standing and sitting.
• Figure 4: Visualization of quantitative evaluation.
• Figure 5: Visualization of the standing aid scenario.