Tactile Recognition of Both Shapes and Materials with Automatic Feature Optimization-Enabled Meta Learning

TL;DR

An automatic feature optimization-enabled prototypical network to realize meta-learning, i.e., AFOP-ML framework is proposed, which not only adapts to new unseen classes rapidly with few-shot, but also learns how to determine the optimal feature space automatically.

Abstract

Tactile perception is indispensable for robots to implement various manipulations dexterously, especially in contact-rich scenarios. However, alongside the development of deep learning techniques, it meanwhile suffers from training data scarcity and a time-consuming learning process in practical applications since the collection of a large amount of tactile data is costly and sometimes even impossible. Hence, we propose an automatic feature optimization-enabled prototypical network to realize meta-learning, i.e., AFOP-ML framework. As a ``learn to learn" network, it not only adapts to new unseen classes rapidly with few-shot, but also learns how to determine the optimal feature space automatically. Based on the four-channel signals acquired from a tactile finger, both shapes and materials are recognized. On a 36-category benchmark, it outperforms several existing approaches by attaining an accuracy of 96.08% in 5-way-1-shot scenario, where only 1 example is available for training. It still remains 88.7% in the extreme 36-way-1-shot case. The generalization ability is further validated through three groups of experiment involving unseen shapes, materials and force/speed perturbations. More insights are additionally provided by this work for the interpretation of recognition tasks and improved design of tactile sensors.

Figures (7)

• Figure 1: Conceptual overview of the proposed AFOP-ML framework. Multi-channel tactile signals produced by a tactile finger are firstly leveraged to learn how to optimize the feature space automatically and predict correct shape and material category as a source task. Then, these knowledge is transferred to the AFOP-ML model to recognize new shapes and materials under normal or changed experimental conditions as a target task with few-shot data available. Features that are fed into the prototypical network are automatically determined for different recognition tasks.
• Figure 2: Tactile sensor and 36 categories to recognize. (a) Bio-inspired tactile finger mainly consists of rigid phalanx, soft skin, PDMS support, fingerprint, fixture and two types of SEs: PVDFs and SGs. Two PVDFs and two SGs constitute Channel 1 to 4. (b) The 36 categories to recognize. Three materials: Resin, Wood and Aluminum. There are 12 shapes for each material.
• Figure 3: The framework of proposed AFOP-ML algorithm. (a) Offline feature selection. Four synchronous channels are converted to a 386-dimensionality feature pool per trial. (b) Episode-time adaptation. For each $N$-way-$K$-shot task, support sets are mapped to Top-$D$ and averaged into class prototypes $W$. Prototypes and queries are $\ell_2$-normalized; cosine similarities are temperature-scaled ($\alpha$) and shifted by class biases $b$, then fed to Softmax. Cross-entropy with entropy regularization updates only $(W,b)$ on the support set; queries are forward-only. (c) Features in frequency domain (192-D). (d) Features in time domain (194-D).
• Figure 4: Automatic determination of the optimal dimensionality of feature space, $D$. 5‑way-5‑shot accuracy vs. the number of selected features (episodic mean ± 95% confidence intervals (CI) across splits).
• Figure 5: Generalization performance in 1-shot task. (a) Cross‑shape, (b) Cross‑material, and (c) Force–speed perturbations. Shaded bands show the variations within 95% CI across splits (no split for perturbations).