"You'll Be Alice Adventuring in Wonderland!" Processes, Challenges, and Opportunities of Creating Animated Virtual Reality Stories

TL;DR

This study investigates how animated VR stories are created and the unique challenges they face by conducting semi-structured interviews with 21 creators. It identifies ten end-to-end production stages, reveals two workflow types (story-driven and visual-driven), and highlights seven challenges comprising seventeen issues, nine of which are unique to animated VR storytelling. Through validation with 15 additional creators, the authors demonstrate consensus on the importance and difficulty of these stages and issues, and offer a set of future research directions, including narrative autonomy, multi-element plot representations, and computational design support for visuals and evaluation. The findings inform both the design of authoring tools for VR narratives and broader directions for HCI research on immersive storytelling, with practical implications for creators seeking streamlined workflows and robust evaluation methods.

Abstract

Animated virtual reality (VR) stories, combining the presence of VR and the artistry of computer animation, offer a compelling way to deliver messages and evoke emotions. Motivated by the growing demand for immersive narrative experiences, more creators are creating animated VR stories. However, a holistic understanding of their creation processes and challenges involved in crafting these stories is still limited. Based on semi-structured interviews with 21 animated VR story creators, we identify ten common stages in their end-to-end creation processes, ranging from idea generation to evaluation, which form diverse workflows that are story-driven or visual-driven. Additionally, we highlight nine unique issues that arise during the creation process, such as a lack of reference material for multi-element plots, the absence of specific functionalities for story integration, and inadequate support for audience evaluation. We compare the creation of animated VR stories to general XR applications and distill several future research opportunities.

Figures (24)

• Figure 1: An illustration of 9 stages in creating an animated VR story (© Coin's team), where the red lines indicate how a storyline evolves. (A) Idea generation: brainstorming general ideas. (B) Story creation: conceiving the main character's activities using sketches (B1) and developing the story by including other characters with scattered word pieces (B2). (C) Scriptwriting: transforming the stories into textual scripts. (D) Storyboarding: sketching the key moments while configuring the timing, camera setups & visual cues. (E) Design: specifying visual (e.g., character appearance) & auditory (e.g., music) elements. (F) Asset development: developing rigged and animated 3D models & audio clips. (G) Scene assembly: assembling individual assets spatially into scenes in the story. (H) Story integration: aligning multiple story elements on the timeline and integrating the whole story. (I) Evaluation: collecting feedback from viewers.
• Figure 2: Various practices and purposes in storyboarding. (A) A high-fidelity storyboard for post-hoc refinement (© AMAO). The creator inspected the compact overview of three world settings (A1-A3) and directly inserted placeholders or added sketches on areas to enhance. (B) A storyboard with interconnected nodes detailing the entire storyline for team communication (© Hedi's team). (C) A storyboard in two forms by a solo creator that selectively captures some moments for self-evaluation (© Ocean Hu).
• Figure 3: Examples of previs to assess scenes and the functionality of interactions before developing high-fidelity models. (A) Low-fidelity models in the previs stage and high-fidelity scenes in the ultimate story (© Hedi's team). (B) Interactive elements that (B1) started as sketches, (B2) evolved into low-fidelity prototypes, and (B3) were incorporated into the final high-fidelity story (B1-B3 © Coin's team).
• Figure 4: An example of interaction design while connecting both the story plots and scenes (© Coin's team). (A) Brainstorming on a whiteboard about possible interactions with available objects within a scene. (B) Marking places where interactions will be triggered on a scene image. (C) Representing interactions in a fishbone diagram to examine whether these interactions can propel the plots forward.
• Figure 5: Key considerations in the scene assembly stage. (A) Optimizing the coherence of 3D spatial relationships amongst assets across varied camera angles (© Jiaming's team). (B) Calibrating light and color settings to convey subtle ambiance (© Tiemu). (C-E): Guiding viewers' attention by (C) enlarging the subject of interest, (D) creating color contrast, and (E) integrating sound effects (C-E © AMAO).