Integrated Control of Robotic Arm through EMG and Speech: Decision-Driven Multimodal Data Fusion

TL;DR

This work tackles enabling hands free control of a robotic arm by fusing EMG and speech modalities. It adopts a decision driven multimodal data fusion framework that leverages modality specific classifiers and a fusion strategy to disambiguate inputs. A hardware testbed using a MYO armband and an Arduino based arm demonstrates that fusion improves accuracy to around 90.6% with reduced error rates to about 5% for fused inputs. The study highlights practical accessibility gains and outlines future extensions to natural language command processing via cloud based speech APIs.

Abstract

Interactions with electronic devices are changing in our daily lives. The day-to-day development brings curiosity to recent technology and challenges its use. The gadgets are becoming cumbersome, and their usage frustrates a segment of society. In specific scenarios, the user cannot use the modalities because of the challenges that bring in, e.g., the usage of touch screen devices by elderly people. The idea of multimodality provides easy access to devices of daily use through various modalities. In this paper, we suggest a solution that allows the operation of a microcontroller-based device using voice and speech. The model implemented will learn from the user's behavior and decide based on prior knowledge.

Figures (6)

• Figure 3: MYO Arm Band
• Figure 4: Arduino based robotic arm.
• Figure 5: Logitech Wireless Headset H800
• Figure 6: MYO Gestures
• Figure 7: Split Out Validation Dataset for Gaussian NB