Design Frameworks for Spatial Zone Agents in XRI Metaverse Smart Environments

TL;DR

This paper addresses the metaverse disconnect between physical space and virtual interfaces by proposing XRI Zone Agents that operate as embodied, context-aware applications within smart spaces. It introduces a design framework that fuses XRI/MiRA theory with spatial computing to define three design axes—mixed reality, agency level, and physical-remote interaction—and demonstrates how zone agents can orchestrate actions across physical IoT devices and virtual representations. A lab-scale prototype shows a plant-avatar lab assistant guiding users through learning, relaxing, and meeting zones via gestures and IoT interactions, illustrating the feasibility of space-as-agent in immersive environments. The work sets the stage for future research on scalable agent-based metaverse spaces, with planned refinements in agency, cross-domain interaction, and richer spatial interaction modalities.

Abstract

The spatial XR-IoT (XRI) Zone Agents concept combines Extended Reality (XR), the Internet of Things (IoT), and spatial computing concepts to create hyper-connected spaces for metaverse applications; envisioning space as zones that are social, smart, scalable, expressive, and agent-based. These zone agents serve as applications and agents (partners, assistants, or guides) for users co-living and co-operating together in a shared spatial context. The zone agent concept is toward reducing the gap between the physical environment (space) and the classical two-dimensional user interface, through space-based interactions for future metaverse applications. This integration aims to enrich user engagement with their environments through intuitive and immersive experiences and pave the way for innovative human-machine interaction in smart spaces. Contributions include: i) a theoretical framework for creating XRI zone/space-agents using Mixed-Reality Agents (MiRAs) and XRI theory, ii) agent and scene design for spatial zone agents, and iii) prototype and user interaction design scenario concepts for human-to-space agent relationships in an early immersive smart-space application.

Figures (5)

• Figure 1: XRI smart-spaces Morris2021XRIWorkstation can be transformed into zone-aware spatial agents that have potential to interact with users based on their spatial contexts. This work focuses on design components and strategies for these agents suzuki2022augmented-robot.
• Figure 2: Zone-aware spatial agents can have a range of Physical and Remote interaction abilities and properties, as well as Mixed Reality Agency (MiRAs), as seen in examples 1-5, based on holz2011mirasuzuki2022augmented-robot.
• Figure 3: Zone agents are here designed for spatial context, based on agent design methodology like padgham2002prometheuswooldridge1995intelligent, (including monitoring and responding to user-zone events such as time context, virtual context, IoT context, zone object context, etc.). These agents express embodiment through zone-driven actions that may be virtual or physical. The level of agency can vary according to agent implementation (from simple reflex agents to high-functioning agents).
• Figure 4: The prototype framework for the multi-agent Zone scenario includes the highlighted actors (user, learning agent, relax agent, meeting agent); these agents interact with the user; which may be both physical, IoT, or virtual interactions, across a communication channel (e.g., MQTT).
• Figure 5: Using the HRI Framework suzuki2022augmented-robot, the design of Zone agents can be considered; showing user interface and widgets, spatial references and visualizations, embedded visual effects, and interaction modalities for an early prototype, as shown. A deeper evaluation of zone agents is left for future work.