Reconfigurable Robot Identification from Motion Data

TL;DR

This work proposes a meta-self-modeling that can deduce robot morphology through proprioception—the robot’s internal sense of its body’s position and movement, and demonstrates the capability of the system to accurately predict robot configurations from proprioceptive signals.

Abstract

Integrating Large Language Models (VLMs) and Vision-Language Models (VLMs) with robotic systems enables robots to process and understand complex natural language instructions and visual information. However, a fundamental challenge remains: for robots to fully capitalize on these advancements, they must have a deep understanding of their physical embodiment. The gap between AI models cognitive capabilities and the understanding of physical embodiment leads to the following question: Can a robot autonomously understand and adapt to its physical form and functionalities through interaction with its environment? This question underscores the transition towards developing self-modeling robots without reliance on external sensory or pre-programmed knowledge about their structure. Here, we propose a meta self modeling that can deduce robot morphology through proprioception (the internal sense of position and movement). Our study introduces a 12 DoF reconfigurable legged robot, accompanied by a diverse dataset of 200k unique configurations, to systematically investigate the relationship between robotic motion and robot morphology. Utilizing a deep neural network model comprising a robot signature encoder and a configuration decoder, we demonstrate the capability of our system to accurately predict robot configurations from proprioceptive signals. This research contributes to the field of robotic self-modeling, aiming to enhance understanding of their physical embodiment and adaptability in real world scenarios.

Figures (7)

• Figure 1: Configuration prediction from motion data. To what degree is it possible to reconstruct the topology of a robot from its motion dynamics alone? (concept illustration only) For more insights into the motivation behind our research, we invite readers to view the supplementary videos.
• Figure 2: A comprehensive view of the reconfigurable robots used in our work. (a) Overview of reconfigurable robots exhibiting a variety of configurations in the simulation environment. (b) Detailed view of the robot's main body, a geometrically precise icosahedron with 20 uniform faces designed for versatile connection to joint modules. (c) Close-up of a single joint module, equipped with an individual motor, demonstrating its potential for connection at 12 distinct angles to allow for a broad range of movement and reconfiguration. (d) Fully-assembled assembled physical robot in the real world.
• Figure 3: Coding method of the icosahedron body and the angle of each link. a) Integer vector coding for the icosahedron body. b)Integer vector coding for the angle of the twelve links. Each face of the icosahedron body is sequentially numbered in a counterclockwise direction from top to bottom. The connected joints allow rotation angles, divided into 12 segments with a 30-degree separation, also numbered in a counterclockwise manner.
• Figure 4: The model architecture of the classifier. The robot signature encoder (a) employs three 1D convolution blocks for channel-wise dependency. Each convolution block (b) employed the squeeze and excitation operation. The spatial features are then encoded through a 2-layer MLP network into a latent vector. Lastly, the latent vector was decoded by seven prediction heads for leg positions and six joints on one side of the robot, where each one is a single fully connected layer. The predicted indices for leg positions and joint angles are selected by taking $argmax$ over each head's output.
• Figure 5: Performance Comparisons. This figure presents bar plots comparing the leg accuracy, average joint accuracy, and total accuracy of our proposed method against the three baselines. The results show that our method outperforms the baselines across all metrics.