Human Arm Pose Estimation with a Shoulder-worn Force-Myography Device for Human-Robot Interaction

TL;DR

This letter investigates the use of a wearable FMG device that can observe the state of the human arm for real-time applications of HRI and proposes a Transformer-based model to map FMG measurements from the shoulder of the user to the physical pose of the arm.

Abstract

Accurate human pose estimation is essential for effective Human-Robot Interaction (HRI). By observing a user's arm movements, robots can respond appropriately, whether it's providing assistance or avoiding collisions. While visual perception offers potential for human pose estimation, it can be hindered by factors like poor lighting or occlusions. Additionally, wearable inertial sensors, though useful, require frequent calibration as they do not provide absolute position information. Force-myography (FMG) is an alternative approach where muscle perturbations are externally measured. It has been used to observe finger movements, but its application to full arm state estimation is unexplored. In this letter, we investigate the use of a wearable FMG device that can observe the state of the human arm for real-time applications of HRI. We propose a Transformer-based model to map FMG measurements from the shoulder of the user to the physical pose of the arm. The model is also shown to be transferable to other users with limited decline in accuracy. Through real-world experiments with a robotic arm, we demonstrate collision avoidance without relying on visual perception.

Figures (9)

• Figure 1: A user is working in a share workspace with a robotic arm. A wearable Force-Myography (FMG) device is used to estimate the pose of the human arm in real time. In this example, when the user reaches to pick up a tool, the robot halts its motion to avoid interference and potential collisions.
• Figure 2: The wearable FMG device includes a back harness and an upper arm band, with a total of 32 FSR sensors. Reflective markers are fixed on the shoulder, elbow and wrist for data collection.
• Figure 3: Illustration of the Transformer-based model for mapping temporal FMG signals to the positions of the elbow and wrist.
• Figure 4: Real and predicted positions of the (top) elbow and (bottom) wrist, with regard to motion time. The mean position errors for the elbow and wrist along the example path are 65.4 mm and 116.6 mm, respectively.
• Figure 5: Position errors of the (top) elbow and (bottom) wrist, with regard to the number of recorded sessions used for training the Transformer-based model. The results show mean and standard deviation values for 15 training attempts while sampling different sessions in each.