Towards Quadrupedal Jumping and Walking for Dynamic Locomotion using Reinforcement Learning

TL;DR

This work addresses the challenge of dynamic quadrupedal locomotion by enabling robust walking alongside precise vertical and horizontal jumping through a curriculum-based reinforcement learning framework. It introduces a reference state initialization (RSI) and projectile-motion-based reward densification to directly train horizontal jumps, eliminating the need for a staged curriculum and improving exploration. Separate LL policies for walking, vertical jumping, and horizontal jumping are learned and validated on the Jumper quadruped, with Sim2Real transfer facilitated by system identification and domain randomization, achieving horizontal jumps up to $1.25\mathrm{m}$ and vertical jumps up to $1.0\mathrm{m}$ with centimeter landing accuracy. The results demonstrate versatile, high-performance dynamic locomotion with potential applications in planetary exploration and complex terrain navigation, including omnidirectional jumping through curriculum adjustments.

Abstract

This paper presents a curriculum-based reinforcement learning framework for training precise and high-performance jumping policies for the robot `Olympus'. Separate policies are developed for vertical and horizontal jumps, leveraging a simple yet effective strategy. First, we densify the inherently sparse jumping reward using the laws of projectile motion. Next, a reference state initialization scheme is employed to accelerate the exploration of dynamic jumping behaviors without reliance on reference trajectories. We also present a walking policy that, when combined with the jumping policies, unlocks versatile and dynamic locomotion capabilities. Comprehensive testing validates walking on varied terrain surfaces and jumping performance that exceeds previous works, effectively crossing the Sim2Real gap. Experimental validation demonstrates horizontal jumps up to 1.25 m with centimeter accuracy and vertical jumps up to 1.0 m. Additionally, we show that with only minor modifications, the proposed method can be used to learn omnidirectional jumping.

Figures (12)

• Figure 1: The Jumper quadruped standing in a Mars analog environment.
• Figure 2: Illustration of Jumper's leg configuration, with key points, components, and angles. This exact leg is the left back leg.
• Figure 3: The robot performing: 1) Jump down from 0.15m, 2) Landing on uneven ground, 3) Vertical jump of 0.75m. 4) Sequential jumps of 0.85m forward and alternating side component of 0.2m, 5) 1.25m jump in simulation. See supplementary video for full sequences, including maximum jumps.
• Figure 4: Control setup in simulation and on robot.
• Figure 5: Examples of RSI for different stages of a jump, standing, in-flight, and touchdown. The red dot indicates the desired landing position.