Physics-Guided Reinforcement Learning System for Realistic Vehicle Active Suspension Control

TL;DR

This work tackles real-time active suspension control under stochastic road input by introducing a physics-guided DRL approach. A Deep Deterministic Policy Gradient (DDPG) agent outputs controllable parameters $k_a$ and $c_a$ within physically feasible bounds to drive a 2-DOF quarter-car model, trained on ISO 8608 road profiles; the policy is constrained to physically meaningful actions and benefits from soft-target updates and experience replay. Quantitatively, the DRL controller significantly reduces mean vertical velocity and acceleration across test roads (roughly 40–50% and up to ~58% respectively) while maintaining smoother, more stable responses than passive suspensions. The method demonstrates improved ride comfort and road-holding with realistic actuation constraints, and the authors provide public code for replication and future extensions to more complex suspension systems.

Abstract

The suspension system is a crucial part of the automotive chassis, improving vehicle ride comfort and isolating passengers from rough road excitation. Unlike passive suspension, which has constant spring and damping coefficients, active suspension incorporates electronic actuators into the system to dynamically control stiffness and damping variables. However, effectively controlling the suspension system poses a challenging task that necessitates real-time adaptability to various road conditions. This paper presents the Physics-Guided Deep Reinforcement Learning (DRL) for adjusting an active suspension system's variable kinematics and compliance properties for a quarter-car model in real time. Specifically, the outputs of the model are defined as actuator stiffness and damping control, which are bound within physically realistic ranges to maintain the system's physical compliance. The proposed model was trained on stochastic road profiles according to ISO 8608 standards to optimize the actuator's control policy. According to qualitative results on simulations, the vehicle body reacts smoothly to various novel real-world road conditions, having a much lower degree of oscillation. These observations mean a higher level of passenger comfort and better vehicle stability. Quantitatively, DRL outperforms passive systems in reducing the average vehicle body velocity and acceleration by 43.58% and 17.22%, respectively, minimizing the vertical movement impacts on the passengers. The code is publicly available at github.com/anh-nn01/RL4Suspension-ICMLA23.

Figures (6)

• Figure 1: Quarter vehicle model with active suspension
• Figure 2: Algorithmic Flow of the DDPG agent Training Pipeline for Active Suspension Control at each time step. Numbers 1 to 4 denote the chronological update order of the networks in each time step. Specifically, the networks shorted by updating order are: (1) Main Policy Network (SGD), (2) Main Q-Network (SGD), (3) Target Policy Network (soft update), and (4) Target Q-Network (soft update).
• Figure 3: Reward Curve of DDPG in each training episode.
• Figure 4: Simple-Bump road: comparative performance of DRL-based Active Controller against Passive Suspension on a simple road bump, modeled with a sine curve. The road bump profile analysis was included for easy verification of the system's physical correctness. Graphs in the first row depict (1) A single hump road excitation and (2) Body Displacement with Passive and DRL Active suspension systems. A smoother displacement curve represents better driving stability. The second row displays the vehicle's Body Velocity (3) and Acceleration (4), respectively, under passive and active DRL system control. A velocity/acceleration closer to 0 represents smoother body movements and a better driving experience. The last two graphs illustrate the dynamic damping stiffness (5) and spring stiffness (6) controlled by DRL.
• Figure 5: Multiple Speed Humps road: comparative performance of DRL-based Active Controller against Passive Suspension. Similar to the simple single-bump profile, this multi-hump road profile is included for intuitive qualitative analysis.