Enabling the Sense of Self in a Dual-Arm Robot

TL;DR

A neural network architecture is implemented to enable a dual-arm robot to differentiate its limbs from the environment using visual and proprioception sensory inputs and demonstrates experimentally that a robot can distinguish itself with an accuracy of 88.7% on average in cluttered environmental settings and under confounding input signals.

Abstract

While humans are aware of their body and capabilities, robots are not. To address this, we present in this paper a neural network architecture that enables a dual-arm robot to get a sense of itself in an environment. Our approach is inspired by human self-awareness developmental levels and serves as the underlying building block for a robot to achieve awareness of itself while carrying out tasks in an environment. We assume that a robot has to know itself before interacting with the environment in order to be able to support different robotic tasks. Hence, we implemented a neural network architecture to enable a robot to differentiate its limbs from the environment using visual and proprioception sensory inputs. We demonstrate experimentally that a robot can distinguish itself with an accuracy of 88.7% on average in cluttered environmental settings and under confounding input signals.

Figures (6)

• Figure 1: A robot differentiates, recognises and situates itself first with its body, and then interacts with the environment.
• Figure 2: The Level 1 architecture combines vision and proprioception inputs of the robot sensors to predict self or environment. As shown above, the model process vision and proprioception through two subnetworks (Resnet18 and Linear layer, respectively), then we concatenate the output features from both subnetworks and then passed to 3 fully connected layers to carry out a prediction.
• Figure 3: Sample images from captured scenes, ref. Table \ref{['tab:four_groups']}
• Figure 4: This images are representing the saliency maps of different environment groups as described in Table \ref{['tab:four_groups']}, were A corresponds to Group-1; B and C, to Group-2; D and F, to Group-3; and, G and H, to Group-4. For each group, the right image shows the predicted label, and the left images shows the regions the model focused on.
• Figure 5: The first row shows the unseen test groups classification accuracies, while the next four rows show the experimental cases for each unseen group.