Immersive and Wearable Thermal Rendering for Augmented Reality

TL;DR

This study addresses the challenge of delivering realistic thermal feedback in augmented reality by formalizing three AR-specific design considerations: indirect feedback to preserve dexterity, thermal passthrough to maintain real-world temperature cues, and spatiotemporal rendering for dynamic sensations. A novel palm-mounted wearable consisting of a 3x3 TEM array with embedded thermistors and integrated water cooling demonstrates closed-loop control and real-time passthrough while allowing natural interaction with real objects. Through four human-subject experiments (N=12), the work shows measurable JNDs for thermal changes, credible passthrough accuracy ($84.6 ext{ extperthousand}$), robust spatial-pattern discrimination, and enhanced immersion, enjoyment, and realism when thermal feedback is present in AR. The findings support the potential of carefully designed thermal devices to bridge physical and virtual interactions in AR, with implications for realism, usability, and future wearable designs.

Abstract

In augmented reality (AR), where digital content is overlaid onto the real world, realistic thermal feedback has been shown to enhance immersion. Yet current thermal feedback devices, heavily influenced by the needs of virtual reality, often hinder physical interactions and are ineffective for immersion in AR. To bridge this gap, we have identified three design considerations relevant for AR thermal feedback: indirect feedback to maintain dexterity, thermal passthrough to preserve real-world temperature perception, and spatiotemporal rendering for dynamic sensations. We then created a unique and innovative thermal feedback device that satisfies these criteria. Human subject experiments assessing perceptual sensitivity, object temperature matching, spatial pattern recognition, and moving thermal stimuli demonstrated the impact of our design, enabling realistic temperature discrimination, virtual object perception, and enhanced immersion. These findings demonstrate that carefully designed thermal feedback systems can bridge the sensory gap between physical and virtual interactions, enhancing AR realism and usability.

Figures (9)

• Figure 1: Overview of the proof-of-concept wearable thermal feedback system. (a) Thermal feedback is applied by an array of individually controlled Peltier devices (thermoelectric modules) with temperature feedback from thermistors mounted to each contact surface. (b) Outward-facing thermistors allow for measurement of contact surface temperatures for thermal passthrough. (c) Interior to the silicone base is a water-cooling channel for temperature management. (d) A single thermal actuation unit consists of a TEM and thermistor mounted on an aluminum base that acts as a water-cooled heatsink. (e) The average temperature response to a step input while in contact with a human palm with an ambient temperature of 30.
• Figure 2: Haptic-mediated thermal interactions in AR enabled by derived design considerations. (a) Physical Interaction. The lack of actuators on the fingers allows for fine manipulation of physical tools or other objects. Direct thermal sensing of contact surfaces is possible due to no occlusion. (b) Physical Interaction. Temperature sensors on the external surface of a haptic device allow collocated thermal actuators on its interior surface to mimic the thermal properties of contacted surfaces. This maintains the thermal sensing capabilities of regions of the body occluded by the device. (c) Virtual Interaction. Thermal sensations applied to nearby occluded regions in sync with the actions of the user can maintain the perception of thermal sensing even when actuators are not present on the fingers. (d) Virtual Interaction. Independently controlled and spatially arranged thermal actuators enable the presentation of sensations that vary both across a contact surface and with time. In this case, the interior surface of a virutal cup being covered and subsequently uncovered by splashes of a warm liquid is represented by a spatially and temporally dynamic sensation on the palm.
• Figure 3: Just Noticeable Difference (JND) Experiment Conditions and Results Two distinct spatial patterns were applied to the device for both heating and cooling JND calculations. (Line) The "Line" pattern activated bottom row of TEMs on the device. (All) For the "All" pattern, all TEMs were activated. (JND Results) The JND (in degrees celsius) quantifies the smallest change in a reference temperature that is detectable with a 75% confidence. Per-participant JND is shown in blue and the overall JND is shown in orange.
• Figure 4: Temperature Passthrough Experimental Setup, Conditions, and Results. (a) A convex aluminum surface with an embedded thermistor sits atop a thermoelectric module and heatsink to act as a temperature-controlled surface. (Table 1) Two conditions--the temperature of the surface relative to ambient and the method of comparison--were tested using a Generalized Linear Mixed Effects model (Table 2).
• Figure 5: Experiment 3 Overview and Results. Transitions between six spatial patterns that utilized the device's 3x3 TEM grid were applied to participants wearing the device, for both warm and cool sensations. The reported discrimination accuracy for each pattern pair transition are shown in the matrix heatmap on the right.