Multi-view Pose Fusion for Occlusion-Aware 3D Human Pose Estimation

TL;DR

This work presents a novel approach for robust 3D human pose estimation in the context of human-robot collaboration, which outperforms state-of-the-art multi-view human pose estimation techniques and demonstrates superior capabilities in handling challenging scenarios with strong occlusions.

Abstract

Robust 3D human pose estimation is crucial to ensure safe and effective human-robot collaboration. Accurate human perception,however, is particularly challenging in these scenarios due to strong occlusions and limited camera viewpoints. Current 3D human pose estimation approaches are rather vulnerable in such conditions. In this work we present a novel approach for robust 3D human pose estimation in the context of human-robot collaboration. Instead of relying on noisy 2D features triangulation, we perform multi-view fusion on 3D skeletons provided by absolute monocular methods. Accurate 3D pose estimation is then obtained via reprojection error optimization, introducing limbs length symmetry constraints. We evaluate our approach on the public dataset Human3.6M and on a novel version Human3.6M-Occluded, derived adding synthetic occlusions on the camera views with the purpose of testing pose estimation algorithms under severe occlusions. We further validate our method on real human-robot collaboration workcells, in which we strongly surpass current 3D human pose estimation methods. Our approach outperforms state-of-the-art multi-view human pose estimation techniques and demonstrates superior capabilities in handling challenging scenarios with strong occlusions, representing a reliable and effective solution for real human-robot collaboration setups.

Figures (6)

• Figure 1: Sample sequence from the Human3.6M-Occluded dataset, with occlusions on three of four views.
• Figure 2: Sample images from the sequences acquired in real human-robot collaboration scenarios. a) Industrial workcell; b) Laboratory workcell.
• Figure 3: Qualitative comparison between our approach and the state of the art on the industrial workcell scenario.
• Figure 4: Qualitative results of our approach and state-of-the-art multi-view methods on the laboratory collaborative workcell.
• Figure 5: Variation of relative and absolute MPJPE with increasing number of de-synchronized cameras. Solid lines represent variation of the relative error, dashed lines represent variation of the absolute errors.