Predicting Multitasking in Manual and Automated Driving with Optimal Supervisory Control

TL;DR

The study addresses driver multitasking in technology-rich vehicles by proposing a hierarchical, computational rational model of optimal supervisory control that allocates visual attention between driving and in-car tasks. Formulated as interconnected continuous-state, continuous-action MDPs with a supervisory agent, the approach predicts context-dependent multitasking across driving demands, road geometries, and automation levels, and is validated against lab and naturalistic data. Key contributions include formalizing multitasking within a $MDP$ framework with a joint reward, demonstrating adaptive gaze strategies (e.g., longer glances on straight roads, longer glances when automation aids are active), and validating predictions on two empirical datasets while releasing open-source code for broader use. The findings have practical implications for designing in-car interfaces and automation to balance safety with information access, informing how driver assistance systems should interact with human attention in diverse driving contexts.

Abstract

Modern driving involves interactive technologies that can divert attention, increasing the risk of accidents. This paper presents a computational cognitive model that simulates human multitasking while driving. Based on optimal supervisory control theory, the model predicts how multitasking adapts to variations in driving demands, interactive tasks, and automation levels. Unlike previous models, it accounts for context-dependent multitasking across different degrees of driving automation. The model predicts longer in-car glances on straight roads and shorter glances during curves. It also anticipates increased glance durations with driver aids such as lane-centering assistance and their interaction with environmental demands. Validated against two empirical datasets, the model offers insights into driver multitasking amid evolving in-car technologies and automation.

Figures (6)

• Figure 1: Our model of multitasking while driving simulates adaptation of task interleaving to internal cognitive constraints and the external environment. A multitasking "supervisor" allocates visual attention either to driving or an in-car visual search task based on expected joint task values. The top panel illustrates how an in-car glance results in increased uncertainty about the driving task, decreasing value predictions of the driving task. Finally, this compels the driver to terminate the visually demanding in-car task and return to the road, halting the progression of the in-car task. The bottom panel contrasts this with a situation where the lane centering assist (LCA) is turned on: due to the LCA helping to keep the lane, the value of the driving subtask decreases more slowly, resulting in a longer in-car glance. The boxplots on the right show aggregated results from our model: LCA allows for an adapted multitasking strategy that results in longer in-car glances and faster task completion.
• Figure 2: Our hierarchical multitasking model consists of a supervisor and two subtask agents. The supervisor allocates visual attention between the driving and in-car subtasks based on the value predictions of the subtask agents. These predictions reflect how well the current subtask states align with goal attainment. The supervisor aims to maximize long-term cumulative joint task performance.
• Figure 3: Changes in the driver agent's value predictions relative to visual attention and driving conditions. Predicted values decline during visual inattention (red) and recover upon re-engaging visual monitoring (green). The effects of driving speed (km/h) and car automation (LCA) are also visible. Notice the impact of LCA under the slower speed: the model predicts that its ability to drive solely under LCA might even be slightly better than with the driver with noisy vision intervening.
• Figure 4: Predicted values of driving and in-car search subtasks (green and blue lines), cumulative reward collected in the episode (red line), and in-car glances (blue background shading) over time. The supervisor monitors dynamically changing subtask values and shifts visual attention accordingly. Driving values are vertically shifted upwards for improved visualization.
• Figure 5: Comparison of our model's predictions and human data in key outcome variables. The x axes labels encode speed (60 or 120 km/h), number or items (6 or 9), and task type (0 = target on screen, 1 = no target on screen).