Dynamic Theater: Location-Based Immersive Dance Theater, Investigating User Guidance and Experience

TL;DR

Dynamic Theater investigates a location-based augmented reality approach to immersive dance, combining three virtual stages, a digital twin for precise dancer placement, and two guidance mechanisms to steer audience exploration. The study captures dancers with a green-screen workflow, renders billowed holographic performances via 4K footage, and uses Azure Spatial Anchors for spatial alignment, enabling natural locomotion through a large indoor space. Results show strong overall enjoyment and a significant overestimation of elapsed time, indicating high immersion, with both particle and arrow guidance contributing to navigation yet differing in intuitiveness. The work demonstrates a practical AR theater blueprint—balancing content placement, guidance, and performer collaboration—that can inform future wide-area AR performances and space-aware choreography.

Abstract

Dynamic Theater explores the use of augmented reality (AR) in immersive theater as a platform for digital dance performances. The project presents a locomotion-based experience that allows for full spatial exploration. A large indoor AR theater space was designed to allow users to freely explore the augmented environment. The curated wide-area experience employs various guidance mechanisms to direct users to the main content zones. Results from our 20-person user study show how users experience the performance piece while using a guidance system. The importance of stage layout, guidance system, and dancer placement in immersive theater experiences are highlighted as they cater to user preferences while enhancing the overall reception of digital content in wide-area AR. Observations after working with dancers and choreographers, as well as their experience and feedback are also discussed.

Figures (10)

• Figure 1: Three virtual stages were captured from the headset using Microsoft MR Capture. Here, both virtual objects and the physical environment are visible within the (a) future stage, (b) fantasy stage, and (c) forest stage.
• Figure 2: A digital twin of the physical space was used to situate the dancers correctly in relation to the environment to preserve their footwork. Image (a) features an illustration shown to the choreographer for precise dancer placement in Unity while (b) shows how dancers were projected through the headset.
• Figure 3: Top view of the experiment area and its physical layout. The dimensions are 208.54$m^{2}$ (2,244 $ft^{2}$).
• Figure 4: (a) The dance capture session employed a green screen setup on a stage measuring 2.9 $m$ x 14.6 $m$, with a green screen backdrop measuring 11 $m$ x 2.6 $m$. (b) Five RealSense D455 cameras, along with a laptop and a 1.2 $m$ tall stand, were positioned between the three DSLR cameras to capture depth video frame data.
• Figure 5: The location-based trigger allowed the viewer to experience the performance as the choreographer intended. The location-based trigger remains invisible and users are unaware of its presence while navigating (a). Upon entering the predetermined trigger spot (b), the dancers emerge (c), often from behind the walls, revealing themselves to the viewers.