Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Design Probes for AI-Driven AAC: Addressing Complex Communication Needs in Aphasia

Lei Mao, Jong Ho Lee, Yasmeen Faroqi Shah, Stephanie Valencia

TL;DR

The paper investigates how AI can address the complex and heterogeneous communication needs of people with aphasia (PWAs) by developing four AI-enhanced AAC design probes evaluated through a two-phase Research through Design study. Eleven PWAs participate in exploratory interviews and interactive evaluations to reveal grounding, trust, and cognitive-load considerations, highlighting benefits and risks of AI for real-time and preparatory communication. The findings show that AI can enhance visual grounding, sentence construction, practice, and storytelling, but misalignment, timing constraints, and context sensitivity remain critical challenges. These insights offer concrete design directions for future AI-driven AAC systems and stress the importance of accommodating aphasia heterogeneity and minimizing user fatigue in design and testing.

Abstract

AI offers key advantages such as instant generation, multi-modal support, and personalized adaptability - potential that can address the highly heterogeneous communication barriers faced by people with aphasia (PWAs). We designed AI-enhanced communication tools and used them as design probes to explore how AI's real-time processing and generation capabilities - across text, image, and audio - can align with PWAs' needs in real-time communication and preparation for future conversations respectively. Through a two-phase "Research through Design" approach, eleven PWAs contributed design insights and evaluated four AI-enhanced prototypes. These prototypes aimed to improve communication grounding and conversational agency through visual verification, grammar construction support, error correction, and reduced language processing load. Despite some challenges, such as occasional mismatches with user intent, findings demonstrate how AI's specific capabilities can be advantageous in addressing PWAs' complex needs. Our work contributes design insights for future Augmentative and Alternative Communication (AAC) systems.

Design Probes for AI-Driven AAC: Addressing Complex Communication Needs in Aphasia

TL;DR

The paper investigates how AI can address the complex and heterogeneous communication needs of people with aphasia (PWAs) by developing four AI-enhanced AAC design probes evaluated through a two-phase Research through Design study. Eleven PWAs participate in exploratory interviews and interactive evaluations to reveal grounding, trust, and cognitive-load considerations, highlighting benefits and risks of AI for real-time and preparatory communication. The findings show that AI can enhance visual grounding, sentence construction, practice, and storytelling, but misalignment, timing constraints, and context sensitivity remain critical challenges. These insights offer concrete design directions for future AI-driven AAC systems and stress the importance of accommodating aphasia heterogeneity and minimizing user fatigue in design and testing.

Abstract

AI offers key advantages such as instant generation, multi-modal support, and personalized adaptability - potential that can address the highly heterogeneous communication barriers faced by people with aphasia (PWAs). We designed AI-enhanced communication tools and used them as design probes to explore how AI's real-time processing and generation capabilities - across text, image, and audio - can align with PWAs' needs in real-time communication and preparation for future conversations respectively. Through a two-phase "Research through Design" approach, eleven PWAs contributed design insights and evaluated four AI-enhanced prototypes. These prototypes aimed to improve communication grounding and conversational agency through visual verification, grammar construction support, error correction, and reduced language processing load. Despite some challenges, such as occasional mismatches with user intent, findings demonstrate how AI's specific capabilities can be advantageous in addressing PWAs' complex needs. Our work contributes design insights for future Augmentative and Alternative Communication (AAC) systems.

Paper Structure

This paper contains 50 sections, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. High-Tech and AI Solutions for Aphasia
  4. Research through Design with People with Aphasia
  5. Design research with Generative AI tools
  6. Phase One: Exploratory Interviews
  7. Method
  8. Interview Findings
  9. Interaction Modality Needs and Expectations.
  10. Current Linguistic and Collaborative Efforts
  11. Timing: Real-time and Practicing for Conversations
  12. Invisible Social Isolation and Negative Emotional Consequences
  13. Cognitive Burden, Sense of Control, and Self-Disclosure
  14. Identified Design Goals
  15. Prototype Design
  16. ...and 35 more sections

Figures (5)

  • Figure 1: System 1 allows users to check and confirm intended words using AI-generated images. Step 1: Users input notes via text or voice. Step 2: AI generates an image and asks for confirmation. Step 3: Users confirm or correct the AI's output, and the final notes are generated.
  • Figure 2: System 2 allows users to generate sentences from keywords and intentions. Step 1: Select intention type (e.g., request, question). Step 2: Input 2–4 keywords. Step 3: Review and confirm AI-generated sentences.
  • Figure 3: System 3 allows users to check and correct the specificity of whole sentences. Step 1: Users input sentences via text or voice. Step 2: AI highlights errors (misspellings, incomplete, context errors). Step 3: Users select corrections, and the final sentence is generated.
  • Figure 4: System 4 allows users to create diary entries by engaging in conversations with an AI chatbot. Step 1: Users input a diary topic and upload photos. Step 2: AI engages in a conversation to collect details. Step 3: System organizes the diary entry into subtopics for future review.
  • Figure 5: An example of a generated diary summary from AP10, a participant with severe aphasia.