Free-form Shape Modeling in XR: A Systematic Review

TL;DR

This state-of-the-art survey addresses free-form shape modeling in XR by focusing on sculpting and 3D sketching within AR/VR/MR contexts. It employs a PRISMA-guided systematic review to classify methods across technique, environment, interaction tools, auto-completion, and collaboration, highlighting AI-assisted approaches and collaborative editing as key trends. The paper provides a taxonomy, synthesizes historical and contemporary methods (from Virtual Clay to neural rendering-based generation), and identifies gaps in tactile feedback, latency, and post-generation editing. The findings offer guidance for designers and researchers on selecting interaction modalities, hardware, and AI-enabled workflows to advance intuitive, collaborative XR shape modeling.

Abstract

Shape modeling research in Computer Graphics has been an active area for decades. The ability to create and edit complex 3D shapes has been of key importance in Computer-Aided Design, Animation, Architecture, and Entertainment. With the growing popularity of Virtual and Augmented Reality, new applications and tools have been developed for artistic content creation; real-time interactive shape modeling has become increasingly important for a continuum of virtual and augmented reality environments (eXtended Reality (XR)). Shape modeling in XR opens new possibilities for intuitive design and shape modeling in an accessible way. Artificial Intelligence (AI) approaches generating shape information from text prompts are set to change how artists create and edit 3D models. There has been a substantial body of research on interactive 3D shape modeling. However, there is no recent extensive review of the existing techniques and what AI shape generation means for shape modeling in interactive XR environments. In this state-of-the-art paper, we fill this research gap in the literature by surveying free-form shape modeling work in XR, with a focus on sculpting and 3D sketching, the most intuitive forms of free-form shape modeling. We classify and discuss these works across five dimensions: contribution of the articles, domain setting, interaction tool, auto-completion, and collaborative designing. The paper concludes by discussing the disconnect between interactive 3D sculpting and sketching and how this will likely evolve with the prevalence of AI shape-generation tools in the future.

Figures (8)

• Figure 1: PRISMA flow diagram for systematic records selection.
• Figure 2: Number of papers published over the years on Sculpting and 3D sketching.
• Figure 3: Number of publications according to the publication venue in our survey duration.
• Figure 4: Different techniques for VR sculpting. (a) Sculpting with hand gestures; (b) & (c) sculpting the model using VR controllers.
• Figure 5: Different techniques of 3D sketching. (a) Sketching on top of a drawing tablet screen; (b) Sketching in AR on a designated area; (c) Drawing in VR.