Uncovering the Metaverse within Everyday Environments: a Coarse-to-Fine Approach

TL;DR

This work tackles uncovering metaverse access points embedded in everyday environments by exploiting Artcode visual markers and a coarse-to-fine vision framework. The proposed VisionGuide pipeline performs heatmap-based coarse localisation followed by a peak-detection step to obtain exact Artcode centres, enabling interactive exploration of virtual worlds. A new SWAD dataset (44 images, 117 Artcodes) and extensive parameter studies (Gaussian filtering, sliding-window sizes, and H-Maxima thresholds) demonstrate robust Artcode localisation across diverse contexts. The approach facilitates aesthetically pleasing, unobtrusive interaction affordances and highlights practical considerations like lighting and deformation, with future work aimed at user studies and performance enhancements.

Abstract

The recent release of the Apple Vision Pro has reignited interest in the metaverse, showcasing the intensified efforts of technology giants in developing platforms and devices to facilitate its growth. As the metaverse continues to proliferate, it is foreseeable that everyday environments will become increasingly saturated with its presence. Consequently, uncovering links to these metaverse items will be a crucial first step to interacting with this new augmented world. In this paper, we address the problem of establishing connections with virtual worlds within everyday environments, especially those that are not readily discernible through direct visual inspection. We introduce a vision-based approach leveraging Artcode visual markers to uncover hidden metaverse links embedded in our ambient surroundings. This approach progressively localises the access points to the metaverse, transitioning from coarse to fine localisation, thus facilitating an exploratory interaction process. Detailed experiments are conducted to study the performance of the proposed approach, demonstrating its effectiveness in Artcode localisation and enabling new interaction opportunities.

Figures (15)

• Figure 1: Visual marker examples.
• Figure 2: Illustration of the framework of the VisionGuide approach.
• Figure 3: Illustration of the sliding window and the moving steps.
• Figure 4: Illustration of the heatmap calculation process and the subsequent peak-finding procedure. The two pink arrows in the left figure indicate the moving direction of the sliding window, which first moves horizontally and then vertically. The blue and green boxes are sliding window examples classified by the trained classifier as containing either non-Artcode or Artcode. In the middle figure, the aggregation results of the left figure are shown, where the darkness level represents the likelihood of each area containing Artcodes---darker areas indicate a higher likelihood of an Artcode presence. The right figure illustrates the procedure for finding regional maxima.
• Figure 5: Input image and its corresponding heatmaps. (\ref{['fig:heatmap-gray']}) and (\ref{['fig:heatmap-color']}) are heatmaps generated by \ref{['algo:calcHeatmap']}, plotted in grayscale and colour space respectively. In these heatmaps, the brighter (higher energy) the area, the higher the likelihood of containing Artcodes. (\ref{['fig:heatmap-fused']}) is the fused result of the gray heatmap with the input image.