Meta-Objects: Interactive and Multisensory Virtual Objects Learned from the Real World for Use in Augmented Reality

TL;DR

The paper proposes meta-objects as next-generation virtual entities that inherit form, properties, and functions from real-world counterparts to enable bidirectional physical–virtual synchronization in a post-metaverse. It defines a three-component architecture—inherited properties, adaptive multisensory feedback, and scene-graph data representation—and outlines a workflow spanning creation (NeRF-based modeling, behavior learning, NFT tokenization), experience (on-device and server-side tracking with real-time multisensory feedback), and an NFT-based economy for creator incentives. The approach leverages offline generation and online modules controlled by a scene-graph intelligence platform to achieve scalable, device-adaptive interactions across wearables and AR/VR systems. This framework promises immersive, realistic cross-realm experiences and a sustainable digital economy by tightly coupling physical properties, multisensory feedback, and real-time synchronization.

Abstract

We introduce the concept of a meta-object, a next-generation virtual object that inherits the form, properties, and functions of its real-world counterpart, enabling seamless synchronization, interaction, and sharing between the physical and virtual worlds. While plenty of today's virtual objects provide some sensory feedback and dynamic behavior, meta-objects fully integrate interactive and multisensory features within a structured data framework to enable real-time immersive experiences in a post-metaverse intelligent simulation platform. Three key components underpin the utilization of meta-objects in the post-metaverse: property-embedded modeling for physical and action realism, adaptive multisensory feedback tailored to user interactions, and a scene graph-based intelligence simulation platform for scalable and efficient ecosystem integration. By leveraging meta-objects through wearable AR/VR devices, the post-metaverse facilitates seamless interactions that transcend spatial and temporal barriers, paving the way for a transformative reality-virtuality convergence.

Figures (5)

• Figure 1: The difference between a meta-object and a virtual object is that it inherits the form, properties, and functions from a corresponding physical object, and it can affect the counterpart physical object. The virtual avatar of the real user could manipulate the real object in remote reality by interacting with a meta-object in the metaverse.
• Figure 2: Three key components of a meta-object with a virtual drone example: (A) Inherited properties, (B) multisensory feedback, and (C) scene graph-based data representation.
• Figure 3: The interaction cycle between users and meta-objects in the post-metaverse. (a) User A (creator) creates a meta-object, (b) User B (consumer) experiences this meta-object, and (c) the economic rewards flow back to User A.
• Figure 4: An overview of the use of meta-objects in the post-metaverse. Gray nodes represent real objects, green nodes represent meta-objects, and blue nodes represent virtual objects. Edges between objects represent semantic relationships, and directed edges between users and objects represent interactions.
• Figure 5: The system diagram for experiencing meta-object in post-metaverse.

Theorems & Definitions (2)

• Definition 1: Meta-Object
• Definition 2: Post-Metaverse