On the Exploration of LM-Based Soft Modular Robot Design

TL;DR

This paper forms the robot design process as a sequence generation task and finds that LLMs are able to capture key requirements expressed in natural language and reflect them in the construction sequences of robots.

Abstract

Recent large language models (LLMs) have demonstrated promising capabilities in modeling real-world knowledge and enhancing knowledge-based generation tasks. In this paper, we further explore the potential of using LLMs to aid in the design of soft modular robots, taking into account both user instructions and physical laws, to reduce the reliance on extensive trial-and-error experiments typically needed to achieve robot designs that meet specific structural or task requirements. Specifically, we formulate the robot design process as a sequence generation task and find that LLMs are able to capture key requirements expressed in natural language and reflect them in the construction sequences of robots. To simplify, rather than conducting real-world experiments to assess design quality, we utilize a simulation tool to provide feedback to the generative model, allowing for iterative improvements without requiring extensive human annotations. Furthermore, we introduce five evaluation metrics to assess the quality of robot designs from multiple angles including task completion and adherence to instructions, supporting an automatic evaluation process. Our model performs well in evaluations for designing soft modular robots with uni- and bi-directional locomotion and stair-descending capabilities, highlighting the potential of using natural language and LLMs for robot design. However, we also observe certain limitations that suggest areas for further improvement.

Figures (7)

• Figure 1: A robot designed by our model with 2.5 feet in contact with the ground
• Figure 2: Module design. (a) A single module is composed of a flexible skeleton, four SMA actuators, and four sphere magnets. (b) An example of real robots built with five modules. (c) A 5-module robot in simulation, where blue lines indicate the springs being actuated, and grey lines indicate the passive SMAs.
• Figure 3: An overview of the data preparation and model training and evaluation pipelines.
• Figure 4: Examples of locomotion for five different robot configurations in both the physical world and simulation. For each row, from left to right: initial configuration in the real world, final configuration after four rounds of gait, initial configuration in the simulation, and final configuration in the simulation.
• Figure 5: Control sequences for various robot configurations: (a) 5-module 2-leg, (b) 5-module 'L'-shape, (c) 4-module 1-chain, (d) 7-module 2-leg, and (e) 8-module 3-leg.