Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Integrating Annotations into the Design Process for Sonifications and Physicalizations

Rhys Sorenson-Graff, S. Sandra Bae, Jordan Wirfs-Brock

TL;DR

The paper addresses the challenge that emergent data representations like sonifications and physicalizations lack conventions for interpretation. It introduces an annotation centered design approach realized through SonNotate/PhysNotate, a card based design probe grounded in the Rahman taxonomy and a research through design methodology. An in person workshop with 13 designers demonstrates that annotations are inseparable from data encodings and can function to bridge creators and audiences and to drive new design ideas. The work contributes by extending existing visualization annotation concepts to sonic and physical modalities and proposing annotation first practices and tools, promoting designing data experiences and outlining cross modal research and social annotation practices.

Abstract

Annotations are a critical component of visualizations, helping viewers interpret the visual representation and highlighting critical data insights. Despite their significant role, we lack an understanding of how annotations can be incorporated into other data representations, such as physicalizations and sonifications. Given the emergent nature of these representations, sonifications, and physicalizations lack formalized conventions (e.g., design space, vocabulary) that can introduce challenges for audiences to interpret the intended data encoding. To address this challenge, this work focuses on how annotations can be more tightly integrated into the design process of creating sonifications and physicalizations. In an exploratory study with 13 designers, we explore how visualization annotation techniques can be adapted to sonic and physical modalities. Our work highlights how annotations for sonification and physicalizations are inseparable from their data encodings.

Integrating Annotations into the Design Process for Sonifications and Physicalizations

TL;DR

The paper addresses the challenge that emergent data representations like sonifications and physicalizations lack conventions for interpretation. It introduces an annotation centered design approach realized through SonNotate/PhysNotate, a card based design probe grounded in the Rahman taxonomy and a research through design methodology. An in person workshop with 13 designers demonstrates that annotations are inseparable from data encodings and can function to bridge creators and audiences and to drive new design ideas. The work contributes by extending existing visualization annotation concepts to sonic and physical modalities and proposing annotation first practices and tools, promoting designing data experiences and outlining cross modal research and social annotation practices.

Abstract

Annotations are a critical component of visualizations, helping viewers interpret the visual representation and highlighting critical data insights. Despite their significant role, we lack an understanding of how annotations can be incorporated into other data representations, such as physicalizations and sonifications. Given the emergent nature of these representations, sonifications, and physicalizations lack formalized conventions (e.g., design space, vocabulary) that can introduce challenges for audiences to interpret the intended data encoding. To address this challenge, this work focuses on how annotations can be more tightly integrated into the design process of creating sonifications and physicalizations. In an exploratory study with 13 designers, we explore how visualization annotation techniques can be adapted to sonic and physical modalities. Our work highlights how annotations for sonification and physicalizations are inseparable from their data encodings.
Paper Structure (12 sections, 2 figures, 1 table)

This paper contains 12 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Annotative Techniques in Visualization
  4. Cards as Playful and Collaborative Design Tools
  5. SonNotate/PhysNotate: An Annotation Game
  6. Game Design Process
  7. Game Components
  8. Exploratory Study
  9. Workshop
  10. Feedback from the Workshop
  11. Future Work and Discussion
  12. Conclusion

Figures (2)

  • Figure 1: Two card sets in SonNotate/PhysNotate: (a) The annotation cards highlight the different techniques (e.g., geometric, connector, text) as well as the purposes, such as adding context or comparing data points. (b) The stakeholder cards include different roles (e.g., parent, athlete) users might embody as well as reasons they might engage with a data representation (e.g., learn, entertain).
  • Figure 2: Two worksheets filled out by participant (P1). (a) "Data Representation" describes a physicalization where boxes of different weights represent carbon footprints of various foods. (b) "Annotation Sketch" where P1 enhanced the physicalization with a geometric annotation: Opening up a box reveals new information to add context.