What Matters for Active Texture Recognition With Vision-Based Tactile Sensors

TL;DR

This paper formalizes the active sampling problem in the context of tactile fabric recognition and provides an implementation of information-theoretic exploration strategies based on minimizing predictive entropy and variance of probabilistic models.

Abstract

This paper explores active sensing strategies that employ vision-based tactile sensors for robotic perception and classification of fabric textures. We formalize the active sampling problem in the context of tactile fabric recognition and provide an implementation of information-theoretic exploration strategies based on minimizing predictive entropy and variance of probabilistic models. Through ablation studies and human experiments, we investigate which components are crucial for quick and reliable texture recognition. Along with the active sampling strategies, we evaluate neural network architectures, representations of uncertainty, influence of data augmentation, and dataset variability. By evaluating our method on a previously published Active Clothing Perception Dataset and on a real robotic system, we establish that the choice of the active exploration strategy has only a minor influence on the recognition accuracy, whereas data augmentation and dropout rate play a significantly larger role. In a comparison study, while humans achieve 66.9% recognition accuracy, our best approach reaches 90.0% in under 5 touches, highlighting that vision-based tactile sensors are highly effective for fabric texture recognition.

Figures (9)

• Figure 1: The texture recognition task requires identifying a given fabric among four comparison samples: (\ref{['subfig:robot_sensor']}) robot arm exploring sample fabrics; (\ref{['subfig:single_fabric']}) dataset of $25$ fabrics; (\ref{['subfig:gelsightimg']}) example tactile image; (\ref{['subfig:human_setup']}) human participant using index fingers to compare fabric samples.
• Figure 2: The considered architectures of the probabilistic classifier: Inception-v3 and small Inception-v3 (Inception-S) with dropout.
• Figure 3: Training and validation accuracy of the Inception-v3 models (see \ref{['subsec:prob_classifier']}) on the non-interactive 25-fabric classification task, averaged over five runs. Each model is trained until the validation accuracy converges. The final validation accuracies are $95.2\%$ for Inception-PT, $94.2\%$ for Inception-RI, and $92.6\%$ for Inception-S.
• Figure 4: Comparing the performance of the Inception-v3 models on the active texture recognition task. Notably, the small Inception network Inception-S performs as well as the larger Inception-PT.
• Figure 5: Comparison of the exploration strategies on the tactile active texture recognition task. Average prediction accuracy, average variance, and entropy of the predictions are shown. Inception-S is used in all experiments. The Variance strategy achieves the highest accuracy, closely followed by Entropy and Random. Interestingly, the Variance strategy leads to a faster entropy decrease even than Entropy (rightmost plot).