Modeling Drivers' Risk Perception via Attention to Improve Driving Assistance

TL;DR

Using this risk formulation to generate FCW alerts may lead to improved false positive rate of FCWs and improved FCW timing, and experiments show that using this risk formulation to generate FCW alerts may lead to improved false positive rate of FCWs and improved FCW timing.

Abstract

Advanced Driver Assistance Systems (ADAS) alert drivers during safety-critical scenarios but often provide superfluous alerts due to a lack of consideration for drivers' knowledge or scene awareness. Modeling these aspects together in a data-driven way is challenging due to the scarcity of critical scenario data with in-cabin driver state and world state recorded together. We explore the benefits of driver modeling in the context of Forward Collision Warning (FCW) systems. Working with real-world video dataset of on-road FCW deployments, we collect observers' subjective validity rating of the deployed alerts. We also annotate participants' gaze-to-objects and extract 3D trajectories of the ego vehicle and other vehicles semi-automatically. We generate a risk estimate of the scene and the drivers' perception in a two step process: First, we model the movement of vehicles in a given scenario as a joint trajectory forecasting problem. Then, we reason about the drivers' risk perception of the scene by counterfactually modifying the input to the forecasting model to represent the drivers' actual observations of vehicles in the scene. The difference in these behaviours gives us an estimate of driver behaviour that accounts for their actual (inattentive) observations and their downstream effect on overall scene risk. We compare both a learned scene representation as well as a more traditional ``worse-case'' deceleration model to achieve the future trajectory forecast. Our experiments show that using this risk formulation to generate FCW alerts may lead to improved false positive rate of FCWs and improved FCW timing.

Figures (5)

• Figure 1: When existing methods (a) seek to use scene risk and driver inattention in conjunction to deploy driver assistance warnings, they are usually separately computed and then combined. We propose a method to incorporate driver inattention into our a scene forecasting model that uses it to hypothesize the driver's scene perception and hence infer risk (b). This forecasting model leads to a more interpretable intermediate trajectory representation and better FCW performance on our dataset.
• Figure 2: Annotation portals for annotating gaze-to-annotation and validity of FCW. See Sec. \ref{['sec:fcw_anno']} for details
• Figure 3: Trajectory extraction from road-facing video. Ego trajectory is extracted via SLAM and non-ego vehicle trajectories are extracted using off-the-shelf tracking and monocular 3D detectors. See Sec. \ref{['sec:traj_ext']} for details.
• Figure 4: Our key assumption: when human drivers have not observed the lead vehicle (blue) for a while, they hypothesize its future dynamics by extrapolating a constant velocity dynamics model. This inaccurate model can lead to the drivers underestimating the scene risk, hence necessitating a warning.
• Figure 5: Results showing conventional vs. gaze aware forward collision warnings for two timesteps within the same epoch. The top left graph shows a bird's eye view (BEV) of the scene. The black icons in BEV represent the current positions of each vehicle, with the $1s$ past (behind) and $3s$ future (ahead) positions. The top right graph shows the same period, but only the longitudinal positions of each vehicle. Inattentive timesteps are represented by white cross marks. The longitudinal graph shows the actual trajectories as well as trajectories hypothesized by the Conventional and Attention-aware FCW. The bottom graph shows the minimum hypothesized future (next $3s$) gap at each timestep. With consecutive timesteps of inattention, the difference between the conventional and attention-aware lead trajectory hypotheses grows, triggering a warning at 4.5 seconds.