Toward Ubiquitous 3D Object Digitization: A Wearable Computing Framework for Non-Invasive Physical Property Acquisition

TL;DR

A wearable and non-invasive computing framework that allows users to conveniently estimate the material elasticity and internal pressure of deformable objects through finger touches is proposed by modeling their local surfaces as pressurized elastic shells and analytically deriving the two physical properties from finger-induced wrinkling patterns.

Abstract

Accurately digitizing physical objects is central to many applications, including virtual/augmented reality, industrial design, and e-commerce. Prior research has demonstrated efficient and faithful reconstruction of objects' geometric shapes and visual appearances, which suffice for digitally representing rigid objects. In comparison, physical properties, such as elasticity and pressure, are also indispensable to the behavioral fidelity of digitized deformable objects. However, existing approaches to acquiring these quantities either rely on invasive specimen collection or expensive/bulky laboratory setups, making them inapplicable to consumer-level usage. To fill this gap, we propose a wearable and non-invasive computing framework that allows users to conveniently estimate the material elasticity and internal pressure of deformable objects through finger touches. This is achieved by modeling their local surfaces as pressurized elastic shells and analytically deriving the two physical properties from finger-induced wrinkling patterns. Together with photogrammetry-reconstructed geometry and textures, the two estimated physical properties enable us to faithfully replicate the motion and deformation behaviors of several deformable objects. For the pressure estimation, our model achieves a relative error of 3.5%. In the interaction experiments, the virtual-physical deformation discrepancy measures less than 10.1%. Generalization to objects of irregular shape further demonstrates the potential of our approach in practical applications. We envision this work to provide insights for and motivate research toward democratizing the ubiquitous and pervasive digitization of our physical surroundings in daily, industrial, and scientific scenarios.

Figures (5)

• Figure 1: The overall workflow of our wearable and non-invasive computational digitization framework. (a) Given a target deformable object, we employ a photogrammetry application available on commercial mobile phones to reconstruct its geometry and textures, as well as a thin-film haptic sensor to measure the finger-exerted indentation force. (b) Using these non-invasively obtained quantities, we analytically compute the surface elastic modulus and internal pressure of the target object based on an inverse computational model. (c) We use all the measured and computed information above to create a faithful digital replica of the target object and interact with it in the virtual environment.
• Figure 2: Estimation of gauge pressure via finger-induced point indentation. (a) Finger-induced point indentation. (b) Linear regression between point force $F$ and indentation depth $w(0)$ for estimating $\hat{P_{g}}$. Each point denotes the mean indentation depth of three trials sharing the same force intensity and the vertical bar shows the standard deviation. The translucent band around a regression line gives the $99\%$ confidence interval for the estimate. (c) Linear regression between $\hat{P_{g}} \approx F/\pi R w(0)$ and $P_{g}$ for estimating $k_{s}$. 'Train' denotes the data used for regressing $k_{s}$; 'Test' denotes the data used for evaluation.
• Figure 3: Evaluation of estimated elastic modulus $E$. (a) Formation of radial wrinkles on a beach ball being poked by fingers. Higher gauge pressure leads to more wrinkles. (b) Experimental setup for controlled vertical point indentation and illustration of the quantities being compared for evaluation. (c) Comparison between real balls deformed by vertical point indentation and their digitized counterparts. The left y-axis of each sub-figure measures the lengths shown by the line plots, while the right y-axis measures the relative errors shown by the bar plots. Each point denotes the mean of three trials sharing the same force intensity, and the translucent band around a line plot shows the standard deviation.
• Figure 4: Dynamic behaviors of a digitized Pezzi ball. (a) Comparison between the maximally deformed shape of a real Pezzi ball undergoing a free fall and its digital counterpart. (b) Comparison between the same Pezzi ball bouncing against the wall and the ground in sequence and its digital counterpart.
• Figure 5: Extension to digitize an irregularly shaped inflated hammer toy. (a) Collection of $\{F, w(0)\}_{i=1}^{N_{1}}$ value pairs via finger-induced point indentation and observation of radial wrinkles. (b) The comparison between swinging the physical toy and its digital counterpart demonstrates spatio-temporally aligned motions and deformations.