Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reinforcement Learning for Wheeled Mobility on Vertically Challenging Terrain

Tong Xu, Chenhui Pan, Xuesu Xiao

TL;DR

This work develops an end-to-end Reinforcement Learning (RL) system for an autonomous vehicle to learn wheeled mobility through simulated trial-and-error experiences that circumvents the need of complex and expensive kinodynamic modeling, planning, and control.

Abstract

Off-road navigation on vertically challenging terrain, involving steep slopes and rugged boulders, presents significant challenges for wheeled robots both at the planning level to achieve smooth collision-free trajectories and at the control level to avoid rolling over or getting stuck. Considering the complex model of wheel-terrain interactions, we develop an end-to-end Reinforcement Learning (RL) system for an autonomous vehicle to learn wheeled mobility through simulated trial-and-error experiences. Using a custom-designed simulator built on the Chrono multi-physics engine, our approach leverages Proximal Policy Optimization (PPO) and a terrain difficulty curriculum to refine a policy based on a reward function to encourage progress towards the goal and penalize excessive roll and pitch angles, which circumvents the need of complex and expensive kinodynamic modeling, planning, and control. Additionally, we present experimental results in the simulator and deploy our approach on a physical Verti-4-Wheeler (V4W) platform, demonstrating that RL can equip conventional wheeled robots with previously unrealized potential of navigating vertically challenging terrain.

Reinforcement Learning for Wheeled Mobility on Vertically Challenging Terrain

TL;DR

This work develops an end-to-end Reinforcement Learning (RL) system for an autonomous vehicle to learn wheeled mobility through simulated trial-and-error experiences that circumvents the need of complex and expensive kinodynamic modeling, planning, and control.

Abstract

Off-road navigation on vertically challenging terrain, involving steep slopes and rugged boulders, presents significant challenges for wheeled robots both at the planning level to achieve smooth collision-free trajectories and at the control level to avoid rolling over or getting stuck. Considering the complex model of wheel-terrain interactions, we develop an end-to-end Reinforcement Learning (RL) system for an autonomous vehicle to learn wheeled mobility through simulated trial-and-error experiences. Using a custom-designed simulator built on the Chrono multi-physics engine, our approach leverages Proximal Policy Optimization (PPO) and a terrain difficulty curriculum to refine a policy based on a reward function to encourage progress towards the goal and penalize excessive roll and pitch angles, which circumvents the need of complex and expensive kinodynamic modeling, planning, and control. Additionally, we present experimental results in the simulator and deploy our approach on a physical Verti-4-Wheeler (V4W) platform, demonstrating that RL can equip conventional wheeled robots with previously unrealized potential of navigating vertically challenging terrain.
Paper Structure (20 sections, 6 equations, 3 figures, 2 tables)

This paper contains 20 sections, 6 equations, 3 figures, 2 tables.

Table of Contents

  1. INTRODUCTION
  2. Related Work
  3. METHOD
  4. VW-Chrono
  5. RL Problem Formulation
  6. State Space
  7. Action Space
  8. Policy Model Architecture
  9. Sliced-Wasserstein Autoencoder (SWAE)
  10. Reward Design
  11. Progress Reward
  12. Rollover Penalty
  13. Timeout Penalty
  14. RESULTS
  15. Baselines
  16. ...and 5 more sections

Figures (3)

  • Figure 1: VW-Chrono: Simulator for Wheeled Mobility on Vertically Challenging Terrain with Increasing Difficulty (Lower Left to Lower Right).
  • Figure 2: Increasing Mobility Difficulty of Vertically Challenging Terrain by Interpolating Start and End with Weight $w$.
  • Figure 3: Custom-Built Testbed with V4W and an Example Trajectory by the RL Algorithm.