Text2Robot: Evolutionary Robot Design from Text Descriptions

TL;DR

Text2Robot presents a manufacturability-aware, text-driven pipeline to design and fabricate quadrupedal robots. By combining text-to-3D mesh generation with geometry-aware conversion to kinetic URDF models and a dual-loop optimization (evolutionary algorithms plus reinforcement learning), the approach rapidly yields functional robots within days. The work demonstrates that strong text-informed initializations improve co-optimization outcomes, enables terrain-adaptive morphologies, and enables sim-to-real transfer with modular assembly. This framework offers a practical path toward rapid, user-driven robotic prototyping and manufacturing at scale.

Abstract

Robot design has traditionally been costly and labor-intensive. Despite advancements in automated processes, it remains challenging to navigate a vast design space while producing physically manufacturable robots. We introduce Text2Robot, a framework that converts user text specifications and performance preferences into physical quadrupedal robots. Within minutes, Text2Robot can use text-to-3D models to provide strong initializations of diverse morphologies. Within a day, our geometric processing algorithms and body-control co-optimization produce a walking robot by explicitly considering real-world electronics and manufacturability. Text2Robot enables rapid prototyping and opens new opportunities for robot design with generative models.

Figures (12)

• Figure 1: Text2Robot creates physical robots from user-specified text prompts and performance preferences while considering real-world electronics and manufacturability.
• Figure 2: Overview of the four steps in Text2Robot framework.
• Figure 3: Geometric Processing. (A) Heat maps to visualize the cross-section area from the XZ (left) and XY plane (right). (B) The selected planes for slicing the mesh model and the coordinate of the center of mass and the resulting robot components and their joint coordinates. (C) The final robot model with extruded boxes for electronics and motors.
• Figure 4: Morphology and Walking Policy Co-optimization. (A) The inner loop implements reinforcement learning to optimize the robot control policy, and the outer loop optimizes the robot morphologies through genetic operations. (B) Our genetic representation and examples of crossover and mutation operation.
• Figure 5: Generated Meshes and Corresponding User Descriptions. (A) Sixteen robot mesh models generated from our structured prompt with diverse user descriptions. (B) We used the same or similar descriptions to generate four other morphology variants for bug, frog, and dog robots.