Exploring the Potential of Robot-Collected Data for Training Gesture Classification Systems

TL;DR

This study investigates training gesture classification systems with robot-collected data as a workaround for scarce human motion samples in smartwatch-based digit recognition. It builds a dataset by recording numeric digits with an Apple Watch IMU and by reproducing the same trajectories on an ABB IRB120 robot using inverse kinematics, then trains a 3-layer MLP on robot data and tests on human data. Results show velocity-derived features achieve the best cross-domain accuracy (~63.7%), indicating feasibility while highlighting remaining gaps compared to human-only training. The work provides a first proof-of-concept for robot-generated training data in human gesture classification and motivates further calibration and alternative classifiers to improve cross-domain performance in data-limited settings.

Abstract

Sensors and Artificial Intelligence (AI) have revolutionized the analysis of human movement, but the scarcity of specific samples presents a significant challenge in training intelligent systems, particularly in the context of diagnosing neurodegenerative diseases. This study investigates the feasibility of utilizing robot-collected data to train classification systems traditionally trained with human-collected data. As a proof of concept, we recorded a database of numeric characters using an ABB robotic arm and an Apple Watch. We compare the classification performance of the trained systems using both human-recorded and robot-recorded data. Our primary objective is to determine the potential for accurate identification of human numeric characters wearing a smartwatch using robotic movement as training data. The findings of this study offer valuable insights into the feasibility of using robot-collected data for training classification systems. This research holds broad implications across various domains that require reliable identification, particularly in scenarios where access to human-specific data is limited.

Figures (2)

• Figure 1: Robot used in the experiments with the smartwatch.
• Figure 2: Comparison between Human and Robot's signals