Active Exploration for Real-Time Haptic Training

TL;DR

This work tackles the challenge of real-time tactile perception learning under the constraints of active sensing and limited data by linking a conditional variational autoencoder to an ergodic, entropy-guided exploration strategy. The closed-loop framework trains a CVAE using BioTac tactile data while steering exploration toward state regions with high model uncertainty, improving data efficiency over random exploration. Six tactile scenes demonstrate that active learning yields lower training losses and yields crisp entropy heatmaps that identify salient scene regions, including edges and corners, even for organic objects. The approach offers a path toward salience-based tactile localization and object identification without hand-crafted features, with practical impact on robust manipulation in vision-denied settings.

Abstract

Tactile perception is important for robotic systems that interact with the world through touch. Touch is an active sense in which tactile measurements depend on the contact properties of an interaction--e.g., velocity, force, acceleration--as well as properties of the sensor and object under test. These dependencies make training tactile perceptual models challenging. Additionally, the effects of limited sensor life and the near-field nature of tactile sensors preclude the practical collection of exhaustive data sets even for fairly simple objects. Active learning provides a mechanism for focusing on only the most informative aspects of an object during data collection. Here we employ an active learning approach that uses a data-driven model's entropy as an uncertainty measure and explore relative to that entropy conditioned on the sensor state variables. Using a coverage-based ergodic controller, we train perceptual models in near-real time. We demonstrate our approach using a biomimentic sensor, exploring "tactile scenes" composed of shapes, textures, and objects. Each learned representation provides a perceptual sensor model for a particular tactile scene. Models trained on actively collected data outperform their randomly collected counterparts in real-time training tests. Additionally, we find that the resulting network entropy maps can be used to identify high salience portions of a tactile scene.

Figures (7)

• Figure 1: Haptic exploration of a leaf on a test token: The high entropy regions of the neural network indicate where the sensor should collect data in the scene---the image edge is the edge of the reachable scene---most relevant to predicting future measurement values. The black ellipse indicates the approximate size of the sensor footprint and the high information content areas include both when the sensor is in direct contact as well as when the sensor is adjacent to the leaf.
• Figure 2: Closed loop data collection for haptic learning: After initially collecting data using a uniform distribution as the specification, the system collects data based on the state of the network while simultaneously training the network based on collected data.
• Figure 3: Network architecture: The haptic measurement is conditioned on the sensor state while the decoder predicts the measurement at any sensor state.
• Figure 4: Sample scenes, exploration trajectory, network entropy: For each data collection episode a sensor exploration trajectory is generated using the network entropy as a target distribution. This process ensures that the sensor spends time in informative areas of the scene.
• Figure 5: Experimental gantry, BioTac sensor and haptic tokens: The interchangeable token each constitute a tactile scene and are held with pins for hot swapping. The sensor's compliant mount can accommodate 6mm of height variation.

Theorems & Definitions (1)

• Definition 1: Ergodic