ProVoice: Designing Proactive Functionality for In-Vehicle Conversational Assistants using Multi-Objective Bayesian Optimization to Enhance Driver Experience

TL;DR

This work tackles the challenge of designing proactive in-vehicle conversational assistants that accommodate diverse driver needs. It introduces ProVoice, a VR driving simulator that employs Human-in-the-Loop Multi-Objective Bayesian Optimization (HITL MOBO) to optimize four continuous design parameters (including Level of Autonomy) against three objectives: minimize mental demand while maximizing predictability and usefulness, using N=19 participants in a within-subjects setup. The study reveals that MOBO-guided designs reduce mental workload and improve perceived predictability and usefulness, with distinct trade-offs between trained versus fixed LoA, and highlights the value of open-source tools to extend proactive IVCA research across scenarios. The findings inform design strategies—emphasizing auditory-first interventions and situational autonomy—and underscore the importance of safety, transparency, and user collaboration in deploying personalized proactive assistance in real-world driving. The work culminates in an open Unity asset and data resources to foster future research and practical deployment.

Abstract

The next step for In-vehicle Conversational Assistants (IVCAs) will be their capability to initiate and automate proactive system interactions throughout journeys. However, diverse drivers make it challenging to design voice interventions tailored towards individual on-road expectations. This paper evaluates the effectiveness of Human-in-the-Loop (HITL) Multi-Objective Bayesian Optimization (MOBO) in design by implementing ProVoice: a Virtual Reality (VR) driving simulator integrating MOBO to investigate the effects of IVCA design variants on perceived mental demand, predictability, and usefulness. By reporting the Pareto Front from a within-subjects VR study (N=19), this paper proposes optimal design trade-offs. Follow-up analysis demonstrates MOBO's success in discovering effective intervention strategies, with reduced participant mental demand, alongside enhanced predictability and usefulness while engaging with the proactive IVCA. Implications for computational techniques in future research on proactive intervention strategies are discussed. ProVoice can extend to include alternative design parameters and driving scenarios, encouraging intervention design on a broad scale.

Figures (14)

• Figure 1: Route overview. White line indicates the full route for one iteration. Golden squares represent potential parking locations.
• Figure 2: Interior vehicle view from the driver cockpit. The blue line represents a support path, defined as a marker to keep drivers on-course to the route boundaries.
• Figure 3: Interior view of the vehicle upon proactive intervention. Each iteration modifies some design parameter for the IVCA.
• Figure 4: Interior questionnaire, presented after locating a final parking destination. Drivers are asked to rate the IVCA variant on mental demand, predictability, and usefulness.
• Figure 5: Exemplar experiment setup. Participants controlled ProVoice with third-party driving equipment and VR headset. The simulator was displayed on a desktop monitor for direct researcher observation.