Situation Awareness for Driver-Centric Driving Style Adaptation

TL;DR

A situation-aware driving style model based on different visual feature encoders pretrained on fleet data, as well as driving behavior predictors, which are adapted to the driving style of a specific driver are proposed.

Abstract

There is evidence that the driving style of an autonomous vehicle is important to increase the acceptance and trust of the passengers. The driving situation has been found to have a significant influence on human driving behavior. However, current driving style models only partially incorporate driving environment information, limiting the alignment between an agent and the given situation. Therefore, we propose a situation-aware driving style model based on different visual feature encoders pretrained on fleet data, as well as driving behavior predictors, which are adapted to the driving style of a specific driver. Our experiments show that the proposed method outperforms static driving styles significantly and forms plausible situation clusters. Furthermore, we found that feature encoders pretrained on our dataset lead to more precise driving behavior modeling. In contrast, feature encoders pretrained supervised and unsupervised on different data sources lead to more specific situation clusters, which can be utilized to constrain and control the driving style adaptation for specific situations. Moreover, in a real-world setting, where driving style adaptation is happening iteratively, we found the MLP-based behavior predictors achieve good performance initially but suffer from catastrophic forgetting. In contrast, behavior predictors based on situationdependent statistics can learn iteratively from continuous data streams by design. Overall, our experiments show that important information for driving behavior prediction is contained within the visual feature encoder. The dataset is publicly available at huggingface.co/datasets/jHaselberger/SADC-Situation-Awareness-for-Driver-Centric-Driving-Style-Adaptation.

Figures (6)

• Figure 1: Distance to lane center predictions of our proposed neural-network-based driving style model (NN) and the driving situation clustering approach (DSC) for two specific scenarios. The top row shows images of the driving situation in chronological order, and the bottom row shows the predicted trajectories and the recorded human behavior. Red squares denote a change in the situation cluster identified by the DSC approach. Corresponding images and their respective cluster IDs are annotated with arrows.
• Figure 2: High-level overview of the proposed method. Our method consists of a visual feature encoder that infers a representation from an image of a driving situation. This encoder is either pretrained on our pretrain dataset, pretrained on ImageNet1K, or a pretrained unsupervised (foundation) model. Utilizing this representation, unsupervised clustering is employed to associate each driving situation with a cluster $C_i$. This clustering can be used to identify and mask specific driving situations to constrain and control the driving style adaptation. We predict the target driving behaviors either with a statistical lookup table that uses the situation cluster $C_i$ for indexing or with MLPs that use the representations from the visual encoder for situation awareness.
• Figure 3: a) Training and validation RMSE of the DSC method on $\mathcal{D}_{V,V}$ for an increasing number of clusters $N_C$ utilizing the ResNeXt-50 feature encoder pretrained on $\mathcal{D}_{P,T}$. b) and c) Predictions of the DSC approach with ResNeXt-50 feature encoding for two specific driving situations with $N_C = 10$ (DSC-10) and $N_C = 500$ (DSC-500).
• Figure 4: Sample images of learned situation clusters using the representations from the visual feature encoders pretrained on our pretrain dataset $\mathcal{D}_{P,T}$, ImageNet1K, and in an unsupervised manner on curated data from different sources. For each situation cluster, we sample six images randomly from the set of assigned driving situations of $\mathcal{D}_{V,T}$. In the first four rows, we aim to highlight various aspects of potential driving situations, including oncoming traffic, following vehicles, overtaking, and driving on rural roads. In the last row, possible shortcomings of the clusters, such as unclear driving situations or over-specification, are shown.
• Figure 5: a) Comparison of the Entropy-based Cluster Specificity ($\mathrm{ECS}$) over the number of clusters $N_C$ of the best-performing models for pretraining variant. b) ECS curves for the models pretrained on our pretrain dataset $\mathcal{D}_{P,T}$.