Design Frameworks for Hyper-Connected Social XRI Immersive Metaverse Environments

TL;DR

The paper tackles the metaverse disconnect problem caused by task-switching across multiple virtual and physical environments by proposing a design framework based on Social XR-IoT (XRI) to create hyper-connected, multi-user metaverse experiences. It develops a three-part design theory—Virtual Embodiment, XRI Interaction, and Agency Design Method—and integrates these into a design architecture that supports shared IoT-enabled environments and avatars across local and remote spaces. Through formal interaction scenarios and a Social-XRI Interaction Cube, the work articulates concrete use-cases and implementation considerations, outlining a path toward prototypes and testbeds for evaluating human-human and human-agent interactions in hybrid spaces. The framework aims to enable remote-work, enhanced connectedness, and multi-agent telepresence in immersive, IoT-enabled metaverses, while acknowledging challenges in integration, privacy, and latency that require future research and prototyping.

Abstract

The metaverse refers to the merger of technologies for providing a digital twin of the real world and the underlying connectivity and interactions for the many kinds of agents within. As this set of technology paradigms - involving artificial intelligence, mixed reality, the internet-of-things and others - gains in scale, maturity, and utility there are rapidly emerging design challenges and new research opportunities. In particular is the metaverse disconnect problem, the gap in task switching that inevitably occurs when a user engages with multiple virtual and physical environments simultaneously. Addressing this gap remains an open issue that affects the user experience and must be overcome to increase overall utility of the metaverse. This article presents design frameworks that consider how to address the metaverse as a hyper-connected meta-environment that connects and expands multiple user environments, modalities, contexts, and the many objects and relationships within them. This article contributes to i) a framing of the metaverse as a social XR-IoT (XRI) concept, ii) design Considerations for XRI metaverse experiences, iii) a design architecture for social multi-user XRI metaverse environments, and iv) descriptive exploration of social interaction scenarios within XRI multi-user metaverses. These contribute a new design framework for metaverse researchers and creators to consider the coming wave of interconnected and immersive smart environments.

Figures (5)

• Figure 1: Existing social metaverse platforms focus on (a) screen and VR experiences and (b) mobile mixed reality and head-mounted experiences. This can be combined with (c) the XRI concept toward a new social-XRI metaverse. Table \ref{['Comparison']} shows more detail and comparison of the systems shown above.
• Figure 2: Design elements for XRI Applications with a focus on virtual embodiment methods, XRI interaction sensing components, and agent system designs, as in holz2011miraguan2022extendedbody.
• Figure 3: Proposed Social XRI Metaverse architecture for single and multiuser, local and remotealmeida2022telepresence, physical and virtual interactions. This involves frameworks for XRI interactionguan2022extendedbody, level of agencywooldridge1995agentholz2011mira, body avatarizationguan2022extendedbody, and level of virtuality across the reality-virtuality continuummilgram1994taxonomy. It defines how users in one or more XRI environments can interact, including their IoT edge devices, sensors, and agents, and the communication methods between them that provide access to shared virtual environments and hybrid objects.
• Figure 4: Design scenarios for local multi-user XRI metaverse interaction wherein two users engage with hybrid virtual-physical IoT objects and XRI avatars, as in XRI Environment 1 (see Figure \ref{['MultipleUserFramework']}). Example interactions include: (a) Manipulating a virtual bulb to turn on/off a physical lamp. (b) Manipulating the scale of a virtual IoT plant avatar. (c) Using computer vision to gather context and interact with virtual agents through physical context changes. (d) Controlling the virtual bulb through conversation. These kinds of interaction are expected to become more common as the metaverse grows in scale and maturity.
• Figure 5: Interactions in the social XRI metaverse are envisioned to scale across multiple environments (see XRI Environment I, II and III), connecting one or more users with diverse sets of IoT-enabled edge devices, agents, and XRI avatars, regardless of their physical or virtual locations, positions across the XRI environment, avatar embodiment, or the access displays used to engage within the metaverse (i.e., screens or HMD's). Example environment configurations are as described above for (a), (b), (c), (d), and (e), as well as the different kinds of interaction (user interaction, IoT interaction, physical presence, virtual telepresence, and traditional display interactions. This level of complex interactions across physical and virtual spaces (single and multiuser, local and remotealmeida2022telepresence, physical and virtual interactions) must be addressed by social XRI metaverse systems. The Social-XRI Interaction Cube (bottom right) shows multiple dimensions and highlights where the context situations (a),(b),(c),(d), and (e) fit within the dimensions.