Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Inclusive External Human-Machine Interface: Exploring the Effects of Visual and Auditory eHMI for Deaf and Hard-of-Hearing People

Wenge Xu, Foroogh Hajiseyedjavadi, Kurtis Weir, Chukwuemeka Eze, Mark Colley

TL;DR

This paper addresses the exclusion of Deaf and Hard-of-Hearing pedestrians from eHMI research for automated vehicles. It systematically combines a formative focus-group study with a VR-based crossing experiment to evaluate visual (No Visual, Abstract Light, Abstract Light + Text, Abstract Light + Symbol) and auditory (Speech vs Without Speech) eHMIs across Hearing and DHH groups. Key findings show that visual eHMIs improve subjective experience and crossing performance, while speech-based auditory cues enhance trust and perceived safety but may not uniformly affect crossing behavior, with DHH participants showing distinct gaze patterns. The study proposes five practical implications to advance inclusive eHMI design and highlights limitations and avenues for future work, including broader populations and more complex scenarios.

Abstract

External Human-Machine Interfaces (eHMIs) have been proposed to facilitate communication between Automated Vehicles (AVs) and pedestrians. However, no attention was given to Deaf and Hard-of-Hearing (DHH) people. We conducted a formative study through focus groups with 6 DHH people and 6 key stakeholders (including researchers, assistive technologists, and automotive interface designers) to compare proposed eHMIs and extract key design requirements. Subsequently, we investigated the effects of visual and auditory eHMI in a virtual reality user study with 32 participants (16 DHH). Results from our scenario suggesting that (1) DHH participants spent more time looking at the AV; (2) both visual and auditory eHMIs enhanced trust, usefulness, and perceived safety; and (3) only visual eHMIs reduced the time to step into the road, time looking at the AV, gaze time, and percentage looking at active visual eHMI components. Lastly, we provided five practical implications for making eHMI inclusive of DHH people.

Towards Inclusive External Human-Machine Interface: Exploring the Effects of Visual and Auditory eHMI for Deaf and Hard-of-Hearing People

TL;DR

This paper addresses the exclusion of Deaf and Hard-of-Hearing pedestrians from eHMI research for automated vehicles. It systematically combines a formative focus-group study with a VR-based crossing experiment to evaluate visual (No Visual, Abstract Light, Abstract Light + Text, Abstract Light + Symbol) and auditory (Speech vs Without Speech) eHMIs across Hearing and DHH groups. Key findings show that visual eHMIs improve subjective experience and crossing performance, while speech-based auditory cues enhance trust and perceived safety but may not uniformly affect crossing behavior, with DHH participants showing distinct gaze patterns. The study proposes five practical implications to advance inclusive eHMI design and highlights limitations and avenues for future work, including broader populations and more complex scenarios.

Abstract

External Human-Machine Interfaces (eHMIs) have been proposed to facilitate communication between Automated Vehicles (AVs) and pedestrians. However, no attention was given to Deaf and Hard-of-Hearing (DHH) people. We conducted a formative study through focus groups with 6 DHH people and 6 key stakeholders (including researchers, assistive technologists, and automotive interface designers) to compare proposed eHMIs and extract key design requirements. Subsequently, we investigated the effects of visual and auditory eHMI in a virtual reality user study with 32 participants (16 DHH). Results from our scenario suggesting that (1) DHH participants spent more time looking at the AV; (2) both visual and auditory eHMIs enhanced trust, usefulness, and perceived safety; and (3) only visual eHMIs reduced the time to step into the road, time looking at the AV, gaze time, and percentage looking at active visual eHMI components. Lastly, we provided five practical implications for making eHMI inclusive of DHH people.
Paper Structure (55 sections, 8 figures, 2 tables)

This paper contains 55 sections, 8 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. DHH People and Road Crossing
  4. External Human-Machine Interface
  5. Visual eHMIs
  6. Auditory eHMI
  7. eHMI and Disabilities
  8. Multi-Modal eHMIs
  9. Formative Study
  10. Participants
  11. Apparatus
  12. Evaluated Concepts
  13. Procedure
  14. Quantitative Measurements
  15. Findings
  16. ...and 40 more sections

Figures (8)

  • Figure 1: Setup of the virtual environment: The left shows a top-down view, while the right shows a leftward view. a) The starting position of the participant, which is 4.7 m from the road. b) The 2 m wide grey pavement area, which is used to inform the vehicle of the intention of crossing. c) The 6.8 m wide two-lane road that the participant needed to cross. d) The waypoint (green surface with downward arrow indicated), which is 2.5 m away from the road, is used to proceed with the partition to the next repetition. The human model was disabled in the experiment, placed here as a visual illusion.
  • Figure 2: a) The Manual vehicle and b) the AV used in the study. i) The Light Strip, ii) The Display.
  • Figure 3: The visual eHMI conditions used in the user study, showing when the vehicle is fully stopped: a) No Visual: No added visual effects; b) Abstract Light: the light strip stayed on; c) Text: the light strip stayed on while the display shows "STOPPED"; d) Symbol: the light strip stayed on while the display shows a symbol of "Walking Man".
  • Figure 4: a) Trust and b) Usefulness ratings by Visual × Audio eHMI conditions, separated for DHH and Hearing groups, with and without speech. Error bars show $\pm$SE.
  • Figure 5: a) Satisfying and b) Perceived Safety ratings by Visual × Audio eHMI conditions, separated for DHH and Hearing groups, with and without speech. Error bars show $\pm$SE.
  • ...and 3 more figures