Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Small Talk, Big Impact? LLM-based Conversational Agents to Mitigate Passive Fatigue in Conditional Automated Driving

Lewis Cockram, Yueteng Yu, Jorge Pardo, Xiaomeng Li, Andry Rakotonirainy, Jonny Kuo, Sebastien Demmel, Mike Lenné, Ronald Schroeter

TL;DR

This study investigates whether an LLM-based conversational agent ( Zoe) can mitigate passive fatigue in SAE Level 3 automated driving on a real-world test track. Through a 40-participant field experiment, the CA delivered brief, environment-tied prompts during monotonous driving, collecting in-car video, Karolinska Sleepiness Scale ratings, and post-drive interviews. Results show the CA supports vigilance by reducing sleepiness and enhancing engagement, with nuanced acceptance shaped by three driver archetypes (Safety-First, Entertainment-Seeking, Social-Connection) that map onto CA design archetypes. The work demonstrates the feasibility and safety-conscious potential of adaptive, persona-aware CAs to maintain attention in safety-critical contexts and provides design guidelines for future CA-HMI development in automated mobility.

Abstract

Passive fatigue during conditional automated driving can compromise driver readiness and safety. This paper presents findings from a test-track study with 40 participants in a real-world rural automated driving scenario. In this scenario, a Large Language Model (LLM) based conversational agent (CA) was designed to check in with drivers and re-engage them with their surroundings. Drawing on in-car video recordings, sleepiness ratings and interviews, we analysed how drivers interacted with the agent and how these interactions shaped alertness. Users found the CA helpful for supporting vigilance during passive fatigue. Thematic analysis of acceptability further revealed three user preference profiles that implicate future intention to use CAs. Positioning empirically observed profiles within existing CA archetype frameworks highlights the need for adaptive design sensitive to diverse user groups. This work underscores the potential of CAs as proactive Human-Machine Interface (HMI) interventions, demonstrating how natural language can support context-aware interaction during automated driving.

Small Talk, Big Impact? LLM-based Conversational Agents to Mitigate Passive Fatigue in Conditional Automated Driving

TL;DR

This study investigates whether an LLM-based conversational agent ( Zoe) can mitigate passive fatigue in SAE Level 3 automated driving on a real-world test track. Through a 40-participant field experiment, the CA delivered brief, environment-tied prompts during monotonous driving, collecting in-car video, Karolinska Sleepiness Scale ratings, and post-drive interviews. Results show the CA supports vigilance by reducing sleepiness and enhancing engagement, with nuanced acceptance shaped by three driver archetypes (Safety-First, Entertainment-Seeking, Social-Connection) that map onto CA design archetypes. The work demonstrates the feasibility and safety-conscious potential of adaptive, persona-aware CAs to maintain attention in safety-critical contexts and provides design guidelines for future CA-HMI development in automated mobility.

Abstract

Passive fatigue during conditional automated driving can compromise driver readiness and safety. This paper presents findings from a test-track study with 40 participants in a real-world rural automated driving scenario. In this scenario, a Large Language Model (LLM) based conversational agent (CA) was designed to check in with drivers and re-engage them with their surroundings. Drawing on in-car video recordings, sleepiness ratings and interviews, we analysed how drivers interacted with the agent and how these interactions shaped alertness. Users found the CA helpful for supporting vigilance during passive fatigue. Thematic analysis of acceptability further revealed three user preference profiles that implicate future intention to use CAs. Positioning empirically observed profiles within existing CA archetype frameworks highlights the need for adaptive design sensitive to diverse user groups. This work underscores the potential of CAs as proactive Human-Machine Interface (HMI) interventions, demonstrating how natural language can support context-aware interaction during automated driving.

Paper Structure

This paper contains 32 sections, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Designing against Passive Fatigue in Conditional Automated Driving
  4. Designing Zoe: A Conversational Agent to Combat Passive Fatigue
  5. Method
  6. Participants
  7. Apparatus
  8. Vehicle and SAE Level 3 Automation
  9. Implementation of the Conversational Agent
  10. Side Video Recording
  11. Karolinska Sleepiness Scale (KSS)
  12. Semi-structured Interview
  13. Procedure
  14. Part I – Introduction and Training
  15. Part II – Familiarisation
  16. ...and 17 more sections

Figures (5)

  • Figure 1: a) The iOS App developed for implementing the conversational agent - Zoe; b) The L3 automated vehicle prototype is running on a repetitive closed-road circuit used to induce passive fatigue during automated driving trials.
  • Figure 2: Timeline of the experimental protocol.
  • Figure 3: In-situ fatigue-related behaviours examples from participants in L3 driving context.
  • Figure 4: Examples of passive, relaxed states and active, engaged behaviours during the monotonous drive and when interacting with the conversational agent.
  • Figure 5: Karolinska Sleepiness Scale (KSS) scores at baseline, pre-interaction, and post-interaction.