Ubiquitous Metadata: Design and Fabrication of Embedded Markers for Real-World Object Identification and Interaction

TL;DR

This work tackles the problem of linking the physical and digital worlds by embedding robust, low-cost, and unobtrusive metadata into everyday objects. It introduces three marker categories—natural, structural, and internal—and provides end-to-end fabrication and detection pipelines for each: SensiCut (speckle-based material ID for laser cutting), StructCode (data embedded in fabrication artifacts), and G-ID (distinct object IDs via slicing parameter variations). Complementary chapters on InfraredTags and BrightMarker extend internal-marker and fluorescence-based approaches to invisible tagging and real-time tracking, respectively. Together, these contributions enable pervasive object identification, AR augmentation, and interaction across design, manufacturing, retail, and MR domains, while addressing scalability and privacy considerations for real-world adoption.

Abstract

The convergence of the physical and digital realms has ushered in a new era of immersive experiences and seamless interactions. As the boundaries between the real world and virtual environments blur and result in a "mixed reality," there arises a need for robust and efficient methods to connect physical objects with their virtual counterparts. In this thesis, we present a novel approach to bridging this gap through the design, fabrication, and detection of embedded machine-readable markers. We categorize the proposed marking approaches into three distinct categories: natural markers, structural markers, and internal markers. Natural markers, such as those used in SensiCut, are inherent fingerprints of objects repurposed as machine-readable identifiers, while structural markers, such as StructCode and G-ID, leverage the structural artifacts in objects that emerge during the fabrication process itself. Internal markers, such as InfraredTag and BrightMarker, are embedded inside fabricated objects using specialized materials. Leveraging a combination of methods from computer vision, machine learning, computational imaging, and material science, the presented approaches offer robust and versatile solutions for object identification, tracking, and interaction. These markers, seamlessly integrated into real-world objects, effectively communicate an object's identity, origin, function, and interaction, functioning as gateways to "ubiquitous metadata" - a concept where metadata is embedded into physical objects, similar to metadata in digital files. Across the different chapters, we demonstrate the applications of the presented methods in diverse domains, including product design, manufacturing, retail, logistics, education, entertainment, security, and sustainability.

Figures (68)

• Figure 1: My research vision. Reading metadata embedded unobtrusively into objects and materials in the physical world and reflecting it in the digital world.
• Figure 2: SensiCut. (a) Many laser cutting materials look alike and are hard to distinguish visually by users or regular cameras. Furthermore, there are many hazardous materials that are often confused for safe ones. (b) SensiCut instead senses the sheet’s unique surface structure using speckle imaging and deep learning. This enables us to select the correct machine settings, which prevents material waste and ensures user safety, as well as (c) precisely engrave multi-material objects.
• Figure 3: StructCode embeds data in the fabrication artifacts of laser-cut objects, such as the patterns of (a) living hinges and (b) finger joints, to augment objects with data. Here, the embedded StructCodes allow narration for a painting and status updates for a potted plant, among others.
• Figure 4: G-ID. (a) 3D printed objects inherently possess surface patterns due to the print path. G-ID exploits such features that would normally go unnoticed to identify unique instances of an object. Our mobile app (b) uses image processing to detect them.
• Figure 5: InfraredTags are 2D markers and barcodes embedded unobtrusively into 3D printed objects and can be detected using infrared cameras (top-right images). This allows new applications for (a) identifying and controlling devices in AR interfaces, (b) embedding metadata such as 3D model URLs into objects, and (c) tracking passive objects for tangible interactions.