A Deep Reinforcement Learning Based Motion Cueing Algorithm for Vehicle Driving Simulation

TL;DR

This paper tackles the challenge of achieving realistic motion cues in vehicle driving simulation by replacing hand-crafted MCAs with a deep reinforcement learning approach. Using PPO, an actor–critic ANN learns a motion cueing policy through interaction with a simulated MSP, mapping MSP states to incremental actuator commands while optimizing a reward that encodes perceptual fidelity and workspace constraints. Validation against an optimized washout-based MCA on a standardized double lane change shows the RL-based MCA yields higher fidelity motion signals, better tilt coordination, and more efficient use of the MSP workspace, with false cues substantially reduced. The work demonstrates that a data-driven, real-time ANN-based MCA can outperform traditional methods and lays groundwork for extending the method to broader trajectories and more detailed vestibular modeling.

Abstract

Motion cueing algorithms (MCA) are used to control the movement of motion simulation platforms (MSP) to reproduce the motion perception of a real vehicle driver as accurately as possible without exceeding the limits of the workspace of the MSP. Existing approaches either produce non-optimal results due to filtering, linearization, or simplifications, or the computational time required exceeds the real-time requirements of a closed-loop application. This work presents a new solution to the motion cueing problem, where instead of a human designer specifying the principles of the MCA, an artificial intelligence (AI) learns the optimal motion by trial and error in interaction with the MSP. To achieve this, a well-established deep reinforcement learning (RL) algorithm is applied, where an agent interacts with an environment, allowing him to directly control a simulated MSP to obtain feedback on its performance. The RL algorithm used is proximal policy optimization (PPO), where the value function and the policy corresponding to the control strategy are both learned and mapped in artificial neural networks (ANN). This approach is implemented in Python and the functionality is demonstrated by the practical example of pre-recorded lateral maneuvers. The subsequent validation shows that the RL algorithm is able to learn the control strategy and improve the quality of the immersion compared to an established method. Thereby, the perceived motion signals determined by a model of the vestibular system are more accurately reproduced, and the resources of the MSP are used more economically.

Figures (8)

• Figure 1: The Robotic Motion Simulator at DLR Institute of System Dynamics and Control (a) 2011Bellmann_CONF and the AVES Research Flight Simulator at DLR Institute of Flight Systems (b) 2013Duda.
• Figure 2: Block Diagram showing the structure of the MDP. At each time step, the agent first defines an action $A_t$ and then receives the resulting state $S_t$ and reward $R_t$.
• Figure 3: Plot of the lateral specific force $f_y$ (a) and the angular velocity $\omega_x$(b) around the roll axis acting on the driver while performing a lateral lane change to the left with a displacement of $2.84$ m and an initial driving speed $v_x$ of $8$ m/s. The therefore resulting maximal and minimal specific forces $f_{y,\text{min/max}}$ are $-1.80$ m/$\text{s}^2$ respectively $1.79$ m/$\text{s}^2$.
• Figure 4: Test track and trajectory of a double lane change according to ISO-3888-1:2018 performed with a passenger car ($1.68$ m width) and a driving velocity of $10$ m/s. The red crosses represent traffic cones that mark out the test track and the blue line shows the driving trajectory.
• Figure 5: Plot of the sensed lateral specific force $f_y$ (a) and the sensed angular velocity $\omega_x$(b) around the roll axis acting on the driver while performing a double lane change according to ISO 3888-1:2018 with the DLR ROboMObil.