Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

When Refreshable Tactile Displays Meet Conversational Agents: Investigating Accessible Data Presentation and Analysis with Touch and Speech

Samuel Reinders, Matthew Butler, Ingrid Zukerman, Bongshin Lee, Lizhen Qu, Kim Marriott

TL;DR

This paper addresses accessible data analysis for BLV users by examining a multimodal interface that combines refreshable tactile displays (RTDs) with conversational agents. Using a Wizard-of-Oz study with 11 BLV participants, the authors explore interactions with line charts, bar charts, and isarithmic maps and identify nine interaction patterns whose use depends on task type and tactile experience. The findings show a strong user preference for the RTD+speech combination over either modality alone and highlight how tactile familiarity enhances independent data interpretation, informing design guidelines for interactive tactile visualizations. The work provides a foundation for developing real systems that enable BLV users to engage in autonomous data exploration and analysis.

Abstract

Despite the recent surge of research efforts to make data visualizations accessible to people who are blind or have low vision (BLV), how to support BLV people's data analysis remains an important and challenging question. As refreshable tactile displays (RTDs) become cheaper and conversational agents continue to improve, their combination provides a promising approach to support BLV people's interactive data exploration and analysis. To understand how BLV people would use and react to a system combining an RTD with a conversational agent, we conducted a Wizard-of-Oz study with 11 BLV participants, where they interacted with line charts, bar charts, and isarithmic maps. Our analysis of participants' interactions led to the identification of nine distinct patterns. We also learned that the choice of modalities depended on the type of task and prior experience with tactile graphics, and that participants strongly preferred the combination of RTD and speech to a single modality. In addition, participants with more tactile experience described how tactile images facilitated a deeper engagement with the data and supported independent interpretation. Our findings will inform the design of interfaces for such interactive mixed-modality systems.

When Refreshable Tactile Displays Meet Conversational Agents: Investigating Accessible Data Presentation and Analysis with Touch and Speech

TL;DR

This paper addresses accessible data analysis for BLV users by examining a multimodal interface that combines refreshable tactile displays (RTDs) with conversational agents. Using a Wizard-of-Oz study with 11 BLV participants, the authors explore interactions with line charts, bar charts, and isarithmic maps and identify nine interaction patterns whose use depends on task type and tactile experience. The findings show a strong user preference for the RTD+speech combination over either modality alone and highlight how tactile familiarity enhances independent data interpretation, informing design guidelines for interactive tactile visualizations. The work provides a foundation for developing real systems that enable BLV users to engage in autonomous data exploration and analysis.

Abstract

Despite the recent surge of research efforts to make data visualizations accessible to people who are blind or have low vision (BLV), how to support BLV people's data analysis remains an important and challenging question. As refreshable tactile displays (RTDs) become cheaper and conversational agents continue to improve, their combination provides a promising approach to support BLV people's interactive data exploration and analysis. To understand how BLV people would use and react to a system combining an RTD with a conversational agent, we conducted a Wizard-of-Oz study with 11 BLV participants, where they interacted with line charts, bar charts, and isarithmic maps. Our analysis of participants' interactions led to the identification of nine distinct patterns. We also learned that the choice of modalities depended on the type of task and prior experience with tactile graphics, and that participants strongly preferred the combination of RTD and speech to a single modality. In addition, participants with more tactile experience described how tactile images facilitated a deeper engagement with the data and supported independent interpretation. Our findings will inform the design of interfaces for such interactive mixed-modality systems.
Paper Structure (29 sections, 3 figures, 5 tables)

This paper contains 29 sections, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Accessible Data Visualization
  4. Refreshable Tactile Displays
  5. Multimodal Display of Graphical Information
  6. Wizard-of-Oz Method
  7. User Study -- Methodology
  8. Participants
  9. Materials
  10. Study Setup and Procedure
  11. The Role of the Wizard
  12. Survey and Overview
  13. Familiarization with the System
  14. Interacting with Water and Fire Visualizations
  15. Interview
  16. ...and 14 more sections

Figures (3)

  • Figure 1: Six of the data visualizations used in the user study. For Water, participants could choose between rainfall or water storage data, or both together. The individual Water line graphs could also be presented using bar graphs (e.g., (e)) that represent the same information.
  • Figure 2: Participants interacting with Water and Fire visualizations.
  • Figure 3: Participants' views on a 5-point Likert scale (1: Strongly Disagree and 5: Strongly Agree) regarding the benefits of the RTD plus the conversational agent (CA) compared to the CA alone, the RTD alone, and tactile graphics (TG) alone; the x-axis represents number of participants.