Pretraining-finetuning Framework for Efficient Co-design: A Case Study on Quadruped Robot Parkour

TL;DR

This paper proposes an approach that co-optimizes the mechanical structure and control policy to boost the locomotive prowess of quadruped robots, and introduces a novel pretraining-finetuning framework, which not only guarantees optimal control strategies for each mechanical candidate but also ensures time efficiency.

Abstract

In nature, animals with exceptional locomotion abilities, such as cougars, often possess asymmetric fore and hind legs. This observation inspired us: could optimizing the leg length of quadruped robots endow them with similar locomotive capabilities? In this paper, we propose an approach that co-optimizes the mechanical structure and control policy to boost the locomotive prowess of quadruped robots. Specifically, we introduce a novel pretraining-finetuning framework, which not only guarantees optimal control strategies for each mechanical candidate but also ensures time efficiency. Additionally, we have devised an innovative training method for our pretraining network, integrating spatial domain randomization with regularization methods, markedly improving the network's generalizability. Our experimental results indicate that the proposed pretraining-finetuning framework significantly enhances the overall co-design performance with less time consumption. Moreover, the co-design strategy substantially exceeds the conventional method of independently optimizing control strategies, further improving the robot's locomotive performance and providing an innovative approach to enhancing the extreme parkour capabilities of quadruped robots.

Figures (7)

• Figure 1: Comparison of co-design methods. (a) Designing distinct control strategies tailored to various morphologies. (b) Utilizing a generalized strategy as a substitute. (c) The algorithm we introduce starts by training a generalized model across varying leg lengths, followed by fine-tuning for specific morphology.
• Figure 2: (a) The URDF model of the quadruped robot, where the solid rectangles represent links, and the dashed ellipses represent joints. The black parts denote fixed parameters, while the blue parts denote parameters that need to be changed. (b) The quadruped robot with different scaling parameters.
• Figure 3: Overview of the proposed co-design pipeline. We initially pre-train a control strategy capable of generalizing across various morphologies. Subsequently, we embed the fine-tuning process into the morphological optimization phase, fine-tuning the pre-trained strategy for each set of candidate morphologies. This approach provides more accurate fitness values for morphological optimization. We employed the Proximal Policy Optimization (PPO) algorithm to update our policy, utilizing an asymmetric Actor-Critic architecture.
• Figure 4: The curve of cumulative rewards changes with the training steps.
• Figure 5: Comparison of Fine-tuning and Standard Training. (a) and (b) represent the results under different morphologies.