CreepyCoCreator? Investigating AI Representation Modes for 3D Object Co-Creation in Virtual Reality

TL;DR

This study investigates how AI co-creative contributions are represented in VR by manipulating embodiment, incremental versus immediate visualization, and highlighting within a Wizard-of-Oz 3D object co-creation task. Using a within-subject 8-condition design, it demonstrates that embodied AI enhances perceived AI contribution and communication but can reduce ownership, while highlighting fails to improve predictability and can lower enjoyment, and incremental visualization increases attention to AI actions but may introduce side effects such as longer task times and higher perceived unexpectedness. The results yield actionable design implications for VR co-creative tools, suggesting when to deploy embodied agents, how to visualize AI changes, and how to balance user control and perception of collaboration. Collectively, the work informs how to better align human-AI collaboration in VR object-building with user expectations and satisfaction, advancing practical guidelines for future co-creative VR systems.

Abstract

Generative AI in Virtual Reality offers the potential for collaborative object-building, yet challenges remain in aligning AI contributions with user expectations. In particular, users often struggle to understand and collaborate with AI when its actions are not transparently represented. This paper thus explores the co-creative object-building process through a Wizard-of-Oz study, focusing on how AI can effectively convey its intent to users during object customization in Virtual Reality. Inspired by human-to-human collaboration, we focus on three representation modes: the presence of an embodied avatar, whether the AI's contributions are visualized immediately or incrementally, and whether the areas modified are highlighted in advance. The findings provide insights into how these factors affect user perception and interaction with object-generating AI tools in Virtual Reality as well as satisfaction and ownership of the created objects. The results offer design implications for co-creative world-building systems, aiming to foster more effective and satisfying collaborations between humans and AI in Virtual Reality.

Figures (11)

• Figure 1:
• Figure 2:
• Figure 3:
• Figure 4:
• Figure 5: