Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

STEMNIST: Spiking Tactile Extended MNIST Neuromorphic Dataset

Anubhab Tripathi, Li Gaishan, Zhengnan Fu, Chiara Bartolozzi, Bert E. Shi, Arindam Basu

TL;DR

STEMNIST tackles the lack of large-scale neuromorphic tactile benchmarks by introducing a 35-class alphanumeric dataset (A–Z, 1–9) collected with a 16×16 tactile sensor at 120 Hz, yielding 7,700 samples and 1,005,592 spike events after adaptive encoding. The work provides an end-to-end pipeline from raw pressure frames to event-based spikes, along with EMNIST-like train-test splits and rigorous fairness testing. Baseline evaluations using a CNN and a Spiking Neural Network achieve 90.91% and 89.16% test accuracy, respectively, demonstrating strong discriminability for complex tactile handwriting and highlighting the benefits of dual-channel spike encoding. STEMNIST offers a reproducible, high-fidelity tactile benchmark that enables hardware-aware algorithm development and paves the way for energy-efficient neuromorphic perception in robotics and assistive technologies.

Abstract

Tactile sensing is essential for robotic manipulation, prosthetics and assistive technologies, yet neuromorphic tactile datasets remain limited compared to their visual counterparts. We introduce STEMNIST, a large-scale neuromorphic tactile dataset extending ST-MNIST from 10 digits to 35 alphanumeric classes (uppercase letters A--Z and digits 1--9), providing a challenging benchmark for event-based haptic recognition. The dataset comprises 7,700 samples collected from 34 participants using a custom \(16\times 16\) tactile sensor array operating at 120 Hz, encoded as 1,005,592 spike events through adaptive temporal differentiation. Following EMNIST's visual character recognition protocol, STEMNIST addresses the critical gap between simplified digit classification and real-world tactile interaction scenarios requiring alphanumeric discrimination. Baseline experiments using conventional CNNs (90.91% test accuracy) and spiking neural networks (89.16%) establish performance benchmarks. The dataset's event-based format, unrestricted spatial variability and rich temporal structure makes it suitable for testing neuromorphic hardware and bio-inspired learning algorithms. STEMNIST enables reproducible evaluation of tactile recognition systems and provides a foundation for advancing energy-efficient neuromorphic perception in robotics, biomedical engineering and human-machine interfaces. The dataset, documentation and codes are publicly available to accelerate research in neuromorphic tactile computing.

STEMNIST: Spiking Tactile Extended MNIST Neuromorphic Dataset

TL;DR

STEMNIST tackles the lack of large-scale neuromorphic tactile benchmarks by introducing a 35-class alphanumeric dataset (A–Z, 1–9) collected with a 16×16 tactile sensor at 120 Hz, yielding 7,700 samples and 1,005,592 spike events after adaptive encoding. The work provides an end-to-end pipeline from raw pressure frames to event-based spikes, along with EMNIST-like train-test splits and rigorous fairness testing. Baseline evaluations using a CNN and a Spiking Neural Network achieve 90.91% and 89.16% test accuracy, respectively, demonstrating strong discriminability for complex tactile handwriting and highlighting the benefits of dual-channel spike encoding. STEMNIST offers a reproducible, high-fidelity tactile benchmark that enables hardware-aware algorithm development and paves the way for energy-efficient neuromorphic perception in robotics and assistive technologies.

Abstract

Tactile sensing is essential for robotic manipulation, prosthetics and assistive technologies, yet neuromorphic tactile datasets remain limited compared to their visual counterparts. We introduce STEMNIST, a large-scale neuromorphic tactile dataset extending ST-MNIST from 10 digits to 35 alphanumeric classes (uppercase letters A--Z and digits 1--9), providing a challenging benchmark for event-based haptic recognition. The dataset comprises 7,700 samples collected from 34 participants using a custom tactile sensor array operating at 120 Hz, encoded as 1,005,592 spike events through adaptive temporal differentiation. Following EMNIST's visual character recognition protocol, STEMNIST addresses the critical gap between simplified digit classification and real-world tactile interaction scenarios requiring alphanumeric discrimination. Baseline experiments using conventional CNNs (90.91% test accuracy) and spiking neural networks (89.16%) establish performance benchmarks. The dataset's event-based format, unrestricted spatial variability and rich temporal structure makes it suitable for testing neuromorphic hardware and bio-inspired learning algorithms. STEMNIST enables reproducible evaluation of tactile recognition systems and provides a foundation for advancing energy-efficient neuromorphic perception in robotics, biomedical engineering and human-machine interfaces. The dataset, documentation and codes are publicly available to accelerate research in neuromorphic tactile computing.
Paper Structure (23 sections, 5 equations, 8 figures, 8 tables)

This paper contains 23 sections, 5 equations, 8 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Materials and Methods
  4. Neuromorphic Tactile Sensing Hardware System
  5. Data Collection
  6. Spiking Tactile Extended MNIST Dataset
  7. Spike Generation and Encoding
  8. File Format and Organization
  9. Dataset Properties
  10. Classification
  11. Model Architectures
  12. Convolutional Neural Network (CNN):
  13. Spiking Neural Network (SNN):
  14. Hyperparameter Optimization
  15. Multi-Split Robustness Evaluation
  16. ...and 8 more sections

Figures (8)

  • Figure 1: Comparison of Neuromorphic Spatio-Temporal Datasets.
  • Figure 2: Tactile Sensing Hardware System used to create the dataset comprising a $16\times 16$ tactile sensor array (TSA), readout electronics and FPGA for data transfer to a PC.
  • Figure 3: Schematic of the Readout Hardware.
  • Figure 4: Protocol of Data Collection.
  • Figure 5: Top Row: Evolution of the normalized "tactile image" over time for the letter 'B', derived from accumulated raw pressure frames. Middle Row: Temporal evolution of the spike-based tactile representation, obtained by summing count of ON and OFF spike events over time. Bottom Row: Spike raster plot showing 185 ON events (red) and 178 OFF events (blue), along the 256 taxels in the duration of the 2-second recording. For this sample, the adaptive threshold was $\theta_{\text{sample}} = 9.0$.
  • ...and 3 more figures