Be There, Be Together, Be Streamed! AR Scenic Live-Streaming for an Interactive and Collective Experience
Zeyu Huang, Zuyu Xu, Yuanhao Zhang, Chengzhong Liu, Yanwei Zhao, Chuhan Shi, Jason Chen Zhao, Xiaojuan Ma
TL;DR
This work proposes Augmented Reality Scenic Live-Streaming (ARSLS) to enhance Scenic Live-Streaming (SLS) by overlaying scene-aware AR content that enables real-time, collective interactions tied to local culture. Implemented as a Unity-based pipeline, ARSLS reconstructs fixed-camera scenes (e.g., West Lake) in 3D, aligns AR content with scene geometry, and responds to audience actions (comments, tips) via a remote server. A West Lake prototype introduces five interactive features—Fei Hua Ling, Lotus Spreading, Fish Feeding, Fireworks, and Oiled-Paper Umbrella with Stories—to foster shared experiences and cultural engagement, validated through a qualitative study with 15 participants. Findings highlight the need to balance virtual embodiment with immersion and to tailor interaction mechanisms to sustain participation, suggesting directions for refining ARSLS and expanding its cultural and interactive repertoire for scalable, meaningful scenic streaming.
Abstract
Scenic Live-Streaming (SLS), capturing real-world scenic sites from fixed cameras without streamers, combines scene immersion and the social and real-time characteristics of live-streaming into a unique experience. However, existing SLS affords limited audience interactions to engage them in a collective experience compared to many other live-streaming genres. It is also difficult for SLS to recreate important but intangible constituents of in-person trip experiences, such as cultural activities. To offer a more interactive, engaging, and meaningful experience, we propose ARSLS (Augmented Reality Scenic Live-Streaming). Culturally grounded AR objects with awareness of the live-streamed environment can be overlaid over camera views to provide additional interactive features while maintaining consistency with the live-streamed scene. To explore the design space of this new medium, we developed an ARSLS prototype for a famous landscape in China. A preliminary study (N=15) provided initial insights for ARSLS design.