Alexa, I Wanna See You: Envisioning Smart Home Assistants for the Deaf and Hard-of-Hearing
Tyrone Justin Sta. Maria, Jordan Aiko Deja
TL;DR
The paper addresses the accessibility gap in smart home assistants (SHAs) for Deaf and Hard-of-Hearing (DHH) users, where voice interfaces and screen-based feedback pose barriers. It proposes a design vision that centers AR visual output, multimodal input (including sign language and gestural commands), and context-aware spaces to enable seamless and inclusive interactions. Key contributions include a blueprint for AR-enabled feedback via steerable projectors, diverse input modalities to reduce reliance on spoken wake-words, and the use of environmental sound sensing to augment awareness with visual alerts, along with a discussion of feasibility, privacy, and deployment challenges. The work aims to advance equitable access to SHAs for the DHH community by outlining concrete design directions and emphasizing user-centered validation and robust security considerations.
Abstract
Smart Home Assistants (SHAs) have become ubiquitous in modern households, offering convenience and efficiency through its voice interface. However, for Deaf and Hard-of-Hearing (DHH) individuals, the reliance on auditory and textual feedback through a screen poses significant challenges. Existing solutions primarily focus on sign language input but overlook the need for seamless interaction and feedback modalities. This paper envisions SHAs designed specifically for DHH users, focusing on accessibility and inclusion. We discuss integrating augmented reality (AR) for visual feedback, support for multimodal input, including sign language and gestural commands, and context awareness through sound detection. Our vision highlights the importance of considering the diverse communication needs of the DHH community in developing SHA to ensure equitable access to smart home technology.