Stabilization of vertical motion of a vehicle on bumpy terrain using deep reinforcement learning
Ameya Salvi, John Coleman, Jake Buzhardt, Venkat Krovi, Phanindra Tallapragada
TL;DR
The paper addresses stabilizing a vehicle's vertical motion over uneven terrain when longitudinal velocity control is the primary actuation and suspension options are limited. It adopts a deep deterministic policy gradient (DDPG) approach trained on a half-car surrogate model with terrain preview from camera, and incorporates dynamic reward shaping to emphasize low vertical acceleration while maintaining target speed. Key contributions include a simulation-first RL workflow with terrain-aware observations, three reward shaping strategies, and a Sim2Real deployment showing improved performance on a scaled vehicle. The findings indicate that a function-weighted reward best reduces peak $\ddot{z}$ and RMSE without sacrificing velocity tracking, and that modest hardware fine-tuning improves transfer, yielding a practical velocity band ($0.6$–$1.0$ m/s) for bump traversal. This work demonstrates the viability of velocity modulation via RL to stabilize vertical dynamics in vehicles lacking active suspension and informs integration with perception and planning in autonomous off-road systems.
Abstract
Stabilizing vertical dynamics for on-road and off-road vehicles is an important research area that has been looked at mostly from the point of view of ride comfort. The advent of autonomous vehicles now shifts the focus more towards developing stabilizing techniques from the point of view of onboard proprioceptive and exteroceptive sensors whose real-time measurements influence the performance of an autonomous vehicle. The current solutions to this problem of managing the vertical oscillations usually limit themselves to the realm of active suspension systems without much consideration to modulating the vehicle velocity, which plays an important role by the virtue of the fact that vertical and longitudinal dynamics of a ground vehicle are coupled. The task of stabilizing vertical oscillations for military ground vehicles becomes even more challenging due lack of structured environments, like city roads or highways, in off-road scenarios. Moreover, changes in structural parameters of the vehicle, such as mass (due to changes in vehicle loading), suspension stiffness and damping values can have significant effect on the controller's performance. This demands the need for developing deep learning based control policies, that can take into account an extremely large number of input features and approximate a near optimal control action. In this work, these problems are addressed by training a deep reinforcement learning agent to minimize the vertical acceleration of a scaled vehicle travelling over bumps by controlling its velocity.