Real-Time Forecasting of Driver-Vehicle Dynamics on 3D Roads: a Deep-Learning Framework Leveraging Bayesian Optimisation

TL;DR

A deep-learning framework to model and predict the evolution of the coupled driver-vehicle system dynamics jointly on a complex road geometry based on Long Short-Term Memory autoencoders and a Bayesian optimiser is proposed to tune some significant hyperparameters of the network.

Abstract

Most state-of-the-art works in trajectory forecasting for automotive target predicting the pose and orientation of the agents in the scene. This represents a particularly useful problem, for instance in autonomous driving, but it does not cover a spectrum of applications in control and simulation that require information on vehicle dynamics features other than pose and orientation. Also, multi-step dynamic simulation of complex multibody models does not seem to be a viable solution for real-time long-term prediction, due to the high computational time required. To bridge this gap, we present a deep-learning framework to model and predict the evolution of the coupled driver-vehicle system dynamics jointly on a complex road geometry. It consists of two components. The first, a neural network predictor, is based on Long Short-Term Memory autoencoders and fuses the information on the road geometry and the past driver-vehicle system dynamics to produce context-aware predictions. The second, a Bayesian optimiser, is proposed to tune some significant hyperparameters of the network. These govern the network complexity, as well as the features importance. The result is a self-tunable framework with real-time applicability, which allows the user to specify the features of interest. The approach has been validated with a case study centered on motion cueing algorithms, using a dataset collected during test sessions of a non-professional driver on a dynamic driving simulator. A 3D track with complex geometry has been employed as driving environment to render the prediction task challenging. Finally, the robustness of the neural network to changes in the driver and track was investigated to set guidelines for future works.

Figures (10)

• Figure 1: Schematic representation of the proposed method. A deep neural network is used to forecast the future dynamics of the driver-vehicle system according to user-specified preferences. A 3D road geometry is considered to render the framework context-aware, guaranteeing generalisation on new roads and drivers with few training data. Finally, a Bayesian optimiser monitors the neural network outputs and tunes the prediction framework to guarantee good prediction performance as well as feasible computational times for online applicability.
• Figure 2: Proposed deep neural network. The historical data (past) of the driver commands and vehicle states are encoded by a bi-directional LSTM to simultaneously represent the recent dynamics as well as the long-term context. In the same way, also the future road geometry is encoded by a bi-directional LSTM. The resulting cell states and hidden states of the two encoders are then passed through parallel fully connected layers, to fuse the past dynamics and future road information and generate the initial hidden and cell states for the decoder. The latter consists of an LSTM layer with recursion and a bi-directional LSTM. The output shape is finally obtained using fully connected layers.
• Figure 3: Dynamic driving simulator used to collect the dataset.
• Figure 4: Planimetry of the test track Calabogie used for the validation of the proposed approach. The right and left margins are shown in red and blue, respectively, i.e. the lap is clockwise. The start line is marked in green.
• Figure 5: Frequency content of the longitudinal acceleration $a_X$, lateral acceleration $a_Y$ and yaw rate $\omega_y$, for the original and downsampled signals. The downsampled signals contain approximately $98\%$ of the power of the original ones.