Expressing and Inferring Action Carefulness in Human-to-Robot Handovers

TL;DR

An ecological approach to infer object characteristics from subtle modulations of the natural kinematics occurring during human object manipulation, and takes inspiration from human strategies to shape robot movements to be communica-tive of the object properties while pursuing the action goals.

Abstract

Implicit communication plays such a crucial role during social exchanges that it must be considered for a good experience in human-robot interaction. This work addresses implicit communication associated with the detection of physical properties, transport, and manipulation of objects. We propose an ecological approach to infer object characteristics from subtle modulations of the natural kinematics occurring during human object manipulation. Similarly, we take inspiration from human strategies to shape robot movements to be communicative of the object properties while pursuing the action goals. In a realistic HRI scenario, participants handed over cups - filled with water or empty - to a robotic manipulator that sorted them. We implemented an online classifier to differentiate careful/not careful human movements, associated with the cups' content. We compared our proposed "expressive" controller, which modulates the movements according to the cup filling, against a neutral motion controller. Results show that human kinematics is adjusted during the task, as a function of the cup content, even in reach-to-grasp motion. Moreover, the carefulness during the handover of full cups can be reliably inferred online, well before action completion. Finally, although questionnaires did not reveal explicit preferences from participants, the expressive robot condition improved task efficiency.

Figures (7)

• Figure 1: A human handover of a cup with water to a robotic arm. The robot pours the water into the (orange) bucket and then places the emptied cup into the (blue) drawer. Human is wearing an eye-tracking device, infrared motion tracking markers, and an IMU sensor on the wrist
• Figure 2: Sequence of actions by the human and robot. From left to right: human-to-robot handover of a full cup (\ref{['fig:handover']}); robot pouring the water content into the bucket (\ref{['fig:pour']}); robot placing the emptied cup in the box (\ref{['fig:drop']}). When manipulating an empty cup, after the handover (\ref{['fig:handover']}), the robot directly drops the cup in the blue container (\ref{['fig:drop']})
• Figure 3: Hand mean velocities and standard deviation, in transparency, associated with the reaching (first peak) and transportation phase (second peak) of full and empty cups, in shades of blue and red, respectively. The robot controller used to transport the cup at a later time did not influence the human behavior as a giver
• Figure 4: Carefulness classification in reaching and transport motions for both empty and full cup's motions as careful or not careful behaviors. The accuracy is measured as the number of motions with full cups correctly predicted as careful behaviors and the motions with empty cups correctly predicted as not careful behaviors.
• Figure 5: The classifier output $B$ for empty and full cup's motions over time. For each human transport of either cup, the $B$ reaching 0% or 100% refers to the classification of C (careful) or NC (not careful) behaviors, respectively. The $\mu$ and $\sigma$ are the mean and the $\sim$84% confidence interval for the true positive cases of the binomial distribution.