Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mixing Modes: Active and Passive Integration of Speech, Text, and Visualization for Communicating Data Uncertainty

Chase Stokes, Chelsea Sanker, Bridget Cogley, Vidya Setlur

TL;DR

This work investigates how speech, text, and visualization can be integrated to communicate data uncertainty. By implementing two multimodal prototypes—passive and active—the study explores how users interpret uncertain information in a road-salt decision scenario. A preliminary user study with 20 participants reveals that there is no one-size-fits-all approach, with effectiveness hinging on user preferences and context, underscoring the need for refined, context-aware multimodal strategies. The findings highlight the complementary roles of visualization, narrated guidance, and textual hedges in supporting decision-making under uncertainty, and point toward adaptable interfaces tuned to user expertise and task context.

Abstract

Interpreting uncertain data can be difficult, particularly if the data presentation is complex. We investigate the efficacy of different modalities for representing data and how to combine the strengths of each modality to facilitate the communication of data uncertainty. We implemented two multimodal prototypes to explore the design space of integrating speech, text, and visualization elements. A preliminary evaluation with 20 participants from academic and industry communities demonstrates that there exists no one-size-fits-all approach for uncertainty communication strategies; rather, the effectiveness of conveying uncertain data is intertwined with user preferences and situational context, necessitating a more refined, multimodal strategy for future interface design.

Mixing Modes: Active and Passive Integration of Speech, Text, and Visualization for Communicating Data Uncertainty

TL;DR

This work investigates how speech, text, and visualization can be integrated to communicate data uncertainty. By implementing two multimodal prototypes—passive and active—the study explores how users interpret uncertain information in a road-salt decision scenario. A preliminary user study with 20 participants reveals that there is no one-size-fits-all approach, with effectiveness hinging on user preferences and context, underscoring the need for refined, context-aware multimodal strategies. The findings highlight the complementary roles of visualization, narrated guidance, and textual hedges in supporting decision-making under uncertainty, and point toward adaptable interfaces tuned to user expertise and task context.

Abstract

Interpreting uncertain data can be difficult, particularly if the data presentation is complex. We investigate the efficacy of different modalities for representing data and how to combine the strengths of each modality to facilitate the communication of data uncertainty. We implemented two multimodal prototypes to explore the design space of integrating speech, text, and visualization elements. A preliminary evaluation with 20 participants from academic and industry communities demonstrates that there exists no one-size-fits-all approach for uncertainty communication strategies; rather, the effectiveness of conveying uncertain data is intertwined with user preferences and situational context, necessitating a more refined, multimodal strategy for future interface design.
Paper Structure (16 sections, 2 figures)

This paper contains 16 sections, 2 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Representations of Uncertainty
  4. Multimodal Techniques for Communicating Information
  5. Prototype Implementation
  6. Decision-making Context
  7. System Components
  8. Data Module
  9. Multimodal Modules
  10. Interaction Module
  11. Preliminary User Study
  12. Method
  13. Participants
  14. Study Findings
  15. Discussion and Future Work
  16. ...and 1 more sections

Figures (2)

  • Figure 1: Diagram showing multimodal components and possible user interactions with passive and active interfaces.
  • Figure 2: Insights from comparing active and passive interfaces. Different features may be better suited for specific interfaces, while some user interactions are consistent.