A nonlinear real time capable motion cueing algorithm based on deep reinforcement learning

TL;DR

This work tackles the challenge of real-time motion cueing for serial MSPs with nonlinear workspaces. It introduces a deep reinforcement learning MCA trained with PPO, incorporating full kinematic nonlinearities and joint constraints, and demonstrates real-time performance while maintaining cue quality comparable to or better than established methods. Key contributions include a complete 6-DOF DRL MCA framework, a reward shaping strategy tuned via automated optimization, and a sim-to-real validation on the DLR MSP with driver-in-the-loop. The results highlight DRL MCA’s potential to efficiently utilize MSP kinematic and dynamic capabilities, offering a scalable approach for high-fidelity, real-time motion cues in complex robotic platforms.

Abstract

In motion simulation, motion cueing algorithms are used for the trajectory planning of the motion simulator platform, where workspace limitations prevent direct reproduction of reference trajectories. Strategies such as motion washout, which return the platform to its center, are crucial in these settings. For serial robotic MSPs with highly nonlinear workspaces, it is essential to maximize the efficient utilization of the MSPs kinematic and dynamic capabilities. Traditional approaches, including classical washout filtering and linear model predictive control, fail to consider platform-specific, nonlinear properties, while nonlinear model predictive control, though comprehensive, imposes high computational demands that hinder real-time, pilot-in-the-loop application without further simplification. To overcome these limitations, we introduce a novel approach using deep reinforcement learning for motion cueing, demonstrated here for the first time in a 6-degree-of-freedom setting with full consideration of the MSPs kinematic nonlinearities. Previous work by the authors successfully demonstrated the application of DRL to a simplified 2-DOF setup, which did not consider kinematic or dynamic constraints. This approach has been extended to all 6 DOF by incorporating a complete kinematic model of the MSP into the algorithm, a crucial step for enabling its application on a real motion simulator. The training of the DRL-MCA is based on Proximal Policy Optimization in an actor-critic implementation combined with an automated hyperparameter optimization. After detailing the necessary training framework and the algorithm itself, we provide a comprehensive validation, demonstrating that the DRL MCA achieves competitive performance against established algorithms. Moreover, it generates feasible trajectories by respecting all system constraints and meets all real-time requirements with low...

Figures (6)

• Figure 1: The DLR Robotic Motion Simulator 2011Bellmann_CONFa.
• Figure 2: A proposed taxonomy of existing motion cueing algorithms. As this work is presenting a neural network-based algorithm achieved through reinforcement learning, this subcategory is marked in red.
• Figure 3: Schematic overview of the used training framework. It includes the nonlinear model of the motion platform and the reference trajectory coming from a vehicle simulation.
• Figure 4: The processing steps of the RL-based MCA to generate a simulator joint trajectory from the vehicle dynamics simulation.
• Figure 5: Specific force and angular velocity for the validation trajectory 3. Presented are the values of the reference and the output of the NMPC, CW and DRL MCAs.