Tangible Scenography as a Holistic Design Method for Human-Robot Interaction

TL;DR

This paper addresses the need for holistic, scene-based design in human-robot interaction by introducing Tangible Scenography as a method inspired by theater and scenography. It presents the Tangible Scenography Kit (TaSK), a tangible design artifact, and reports exploratory design sessions with eight professional designers to prototype holistic HRI scenarios that extend beyond narrow task-oriented robot behavior. The study yields four key insights on the design process, strategies, perceptions of TaSK, and facilitation techniques, highlighting TaSK's potential to broaden ideation, enable parallel scene flows, and support narrative storytelling in HRI. The work demonstrates how scene-based design can enrich creative exploration, foster interdisciplinary collaboration, and offer practical guidance for practitioners, albeit with limitations such as reliance on a single expert's input and varied designer familiarity with tangible tools. Overall, tangible scenography emerges as a promising addition to the designer’s toolkit for developing rich, context-aware HRI scenarios, with open-source TaSK intended to enable replication and evolution across domains.

Abstract

Traditional approaches to human-robot interaction design typically examine robot behaviors in controlled environments and narrow tasks. These methods are impractical for designing robots that interact with diverse user groups in complex human environments. Drawing from the field of theater, we present the construct of scenes -- individual environments consisting of specific people, objects, spatial arrangements, and social norms -- and tangible scenography, as a holistic design approach for human-robot interactions. We created a design tool, Tangible Scenography Kit (TaSK), with physical props to aid in design brainstorming. We conducted design sessions with eight professional designers to generate exploratory designs. Designers used tangible scenography and TaSK components to create multiple scenes with specific interaction goals, characterize each scene's social environment, and design scene-specific robot behaviors. From these sessions, we found that this method can encourage designers to think beyond a robot's narrow capabilities and consider how they can facilitate complex social interactions.

Figures (7)

• Figure 1: Methodological Background for Tangible Scenography: Our proposed method contribution (see (d))-- is contextualized within the broader landscape of design methods in HCI, including design thinking and design exploration (see (a) adapted from luria2020robotic); performing arts (see (b) adapted from luria2020robotic); and tangible design toolkits and props (see (c) adapted from porfirio2021figaro)
• Figure 2: Tangible design resources provided in our design artifact, Tangible Scenography Kit The kit included scenic components (backdrops, figurines, and blocks); stage effects (fluid generation machines, spotlights, LED strips, turntable, and sound effects), craft supplies (papers, felt, wooden sticks, pipe cleaners, and stickers), as well as supplemental design prompts (a design brief including five contextual plot options, a conflict for the plot, a sheet illustrating robot embodiment, and a storyboard template).
• Figure 3: The tangible scenography procedure consisted of five steps: (1) Plot and Backdrop Selection (5 mins) (2) Scene design (20-45 mins) (3) Planned Intervention (10-25 mins) to introduce a conflict in the plot (4) Discussion of Robot Embodiment Options (5 mins) (5) Final video recording to communicate the scene (10 mins).
• Figure 4: Materials and procedure for a tangible scenography. (a) study room set up (b) the Tangible Scenography Kit (TaSK) (c) Participant sketching, (d) exploring the kit components, (e) using the kit components to build a scene.
• Figure 5: Participants' final scenarios from scene-based design sessions: [P1] created a scenario for a "friendly neighborhood robot", including a rotating fountain, grass areas, benches, lighting, and a charging station in the park. [P2] created a "story of a teaching robot," depicting a surveillance robot with spotlights for non-verbal communication and pollution from park visitors. [P3] created a scenario to guide elderly residents from indoors to outdoors and meet children at the park for therapy. [P4] created an indoor navigation scenario in an elder care facility, where the robot guides residents and conducts small-talk about decor and bulletin board. [P5] and [P6] used play to motivate children to clean their bedrooms. [P5] created a social play scenario with positive reinforcement and [P6] created competitive and collaborative games.