TangibleNet: Synchronous Network Data Storytelling through Tangible Interactions in Augmented Reality

TL;DR

This work tackles the challenge of synchronous network data storytelling by introducing TangibleNet, a projector based AR system that uses double sided magnets to manipulate node link diagrams in real time. It combines interviews with news anchors and workshops with VIS/HCI researchers to derive a three dimensional design space for interaction commands, primary modalities, and object multiplexity, and validates the approach through a user study with 12 participants. The contributions include a concrete design space, a functional TangibleNet prototype, and empirical evidence that physical object interactions enhance intuition, autonomy, and engagement during live data storytelling. The findings suggest practical guidelines for future systems that blend physical props with network visualizations to support dynamic, audience aware narratives in diverse settings.

Abstract

Synchronous data-driven storytelling with network visualizations presents significant challenges due to the complexity of real-time manipulation of network components. While existing research addresses asynchronous scenarios, there is a lack of effective tools for live presentations. To address this gap, we developed TangibleNet, a projector-based AR prototype that allows presenters to interact with node-link diagrams using double-sided magnets during live presentations. The design process was informed by interviews with professionals experienced in synchronous data storytelling and workshops with 14 HCI/VIS researchers. Insights from the interviews helped identify key design considerations for integrating physical objects as interactive tools in presentation contexts. The workshops contributed to the development of a design space mapping user actions to interaction commands for node-link diagrams. Evaluation with 12 participants confirmed that TangibleNet supports intuitive interactions and enhances presenter autonomy, demonstrating its effectiveness for synchronous network-based data storytelling.

Figures (6)

• Figure 1: Illustrations of news anchors at work: (a) A3 explaining the rise in COVID-19 cases in Tokyo during a news segment. (b) A scene of a news anchor explaining the relationships between politicians using a physical board, as described by A4 and A5. The visuals are revealed in sync with the narration by removing stickers.
• Figure 2: Workshop: Participants used double-sided magnets (A) to explore user actions for visualization commands and recorded their findings in worksheets (B). They referred to the illustration of visualization commands as needed (C). The ideas were then shared and discussed among the participants.
• Figure 3: The design space maps user actions with double-sided magnets to interaction commands for network visualizations. The horizontal axis represents user actions, the vertical axis shows interaction commands, and each cell displays mappings. Icons in the cells indicate magnet multiplexity, as illustrated in the top-right legend. Mappings implemented in our prototype are marked with an asterisk.
• Figure 4: System Overview: TangibleNet recognizes the presenter's interactions with magnets and hand gestures on the whiteboard via a webcam and projects the resulting visualizations onto the whiteboard using a projector. The projected user interface is divided into two areas: (A) storyboard area and (B) registration area.
• Figure 5: Supported interactions include: (1) Attaching or detaching magnets to show or hide nodes; (2) Rotating a magnet to scale a node; (3) Flipping a magnet to change a node's state; (4) Stacking a magnet to highlight a node; (5) Tapping magnets sequentially to show or hide links; (6) Holding magnets simultaneously to change link types; (7) Tapping the source magnet first and then the target magnet to change link direction; (8) Holding one magnet and rotating the other to scale the link; (9) Bringing magnets closer to show or extend a group; (10) Covering a magnet to hide or shrink a group; (11) Pointing a magnet to display an annotation; (12) Rotating a magnet 360° to show or hide a child network.