A Tangible Multi-Display Toolkit to Support the Collaborative Design Exploration of AV-Pedestrian Interfaces

TL;DR

This paper presents a tangible multi-display toolkit to support the early design exploration of AV-pedestrian interfaces, addressing the gap in methods for multi-stakeholder collaboration in cyber-physical systems. The toolkit combines cross-tablet 3D simulations, tangible interaction objects, and a real-time configuration app, enabling multi-perspective viewing and hands-on manipulation of light-based AV signals. Through expert design sessions and a within-subject study, the authors show that the toolkit enhances context understanding and communication, while remaining usable with some initial setup overhead. The work provides a reproducible toolkit description, hardware and software components, and insights that extend beyond AVs to other cyber-physical systems and urban robots, promoting more inclusive and iterative design practices.

Abstract

The advent of cyber-physical systems, such as robots and autonomous vehicles (AVs), brings new opportunities and challenges for the domain of interaction design. Though there is consensus about the value of human-centred development, there is a lack of documented tailored methods and tools for involving multiple stakeholders in design exploration processes. In this paper we present a novel approach using a tangible multi-display toolkit. Orchestrating computer-generated imagery across multiple displays, the toolkit enables multiple viewing angles and perspectives to be captured simultaneously (e.g. top-view, first-person pedestrian view). Participants are able to directly interact with the simulated environment through tangible objects. At the same time, the objects physically simulate the interface's behaviour (e.g. through an integrated LED display). We evaluated the toolkit in design sessions with experts to collect feedback and input on the design of an AV-pedestrian interface. The paper reports on how the combination of tangible objects and multiple displays supports collaborative design explorations.

Figures (6)

• Figure 1: Early concept of the prototyping toolkit to evaluate various light patterns.
• Figure 2: Overview of the various components of the toolkit and its usage (top), implementation details and network communication between the different components (bottom).
• Figure 3: Participants interacting with the tangible multi-display representation (left), and the single-display simulation (right) in a collaborative design exploration session.
• Figure 4: Study procedure and data collection.
• Figure 5: Results of Q1: "The prototyping toolkit facilitated the understanding of the design context" and Q2: "The prototyping toolkit facilitated communication". [SD=Single Display Simulation, TMD=Tangible Multi-Display Toolkit]