Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Random Channel Ablation for Robust Hand Gesture Classification with Multimodal Biosignals

Keshav Bimbraw, Jing Liu, Ye Wang, Toshiaki Koike-Akino

TL;DR

This work addresses robust hand gesture classification from multimodal forearm biosignals (ultrasound and FMG) in the presence of missing channels. It introduces Random Channel Ablation (RChA), a training strategy that randomly masks channels during learning to produce a universal classifier capable of handling up to K missing channels. Across experiments with 2 subjects and 12 gestures, RChA substantially outperforms Baseline and Imputation methods and closely approaches Oracle performance, demonstrating strong robustness to data loss and practical potential for real-world multimodal sensing. The approach also highlights the feasibility of a single all-in-one classifier for varying missing-channel scenarios, with future work envisioned to extend modalities and subject populations.

Abstract

Biosignal-based hand gesture classification is an important component of effective human-machine interaction. For multimodal biosignal sensing, the modalities often face data loss due to missing channels in the data which can adversely affect the gesture classification performance. To make the classifiers robust to missing channels in the data, this paper proposes using Random Channel Ablation (RChA) during the training process. Ultrasound and force myography (FMG) data were acquired from the forearm for 12 hand gestures over 2 subjects. The resulting multimodal data had 16 total channels, 8 for each modality. The proposed method was applied to convolutional neural network architecture, and compared with baseline, imputation, and oracle methods. Using 5-fold cross-validation for the two subjects, on average, 12.2% and 24.5% improvement was observed for gesture classification with up to 4 and 8 missing channels respectively compared to the baseline. Notably, the proposed method is also robust to an increase in the number of missing channels compared to other methods. These results show the efficacy of using random channel ablation to improve classifier robustness for multimodal and multi-channel biosignal-based hand gesture classification.

Random Channel Ablation for Robust Hand Gesture Classification with Multimodal Biosignals

TL;DR

This work addresses robust hand gesture classification from multimodal forearm biosignals (ultrasound and FMG) in the presence of missing channels. It introduces Random Channel Ablation (RChA), a training strategy that randomly masks channels during learning to produce a universal classifier capable of handling up to K missing channels. Across experiments with 2 subjects and 12 gestures, RChA substantially outperforms Baseline and Imputation methods and closely approaches Oracle performance, demonstrating strong robustness to data loss and practical potential for real-world multimodal sensing. The approach also highlights the feasibility of a single all-in-one classifier for varying missing-channel scenarios, with future work envisioned to extend modalities and subject populations.

Abstract

Biosignal-based hand gesture classification is an important component of effective human-machine interaction. For multimodal biosignal sensing, the modalities often face data loss due to missing channels in the data which can adversely affect the gesture classification performance. To make the classifiers robust to missing channels in the data, this paper proposes using Random Channel Ablation (RChA) during the training process. Ultrasound and force myography (FMG) data were acquired from the forearm for 12 hand gestures over 2 subjects. The resulting multimodal data had 16 total channels, 8 for each modality. The proposed method was applied to convolutional neural network architecture, and compared with baseline, imputation, and oracle methods. Using 5-fold cross-validation for the two subjects, on average, 12.2% and 24.5% improvement was observed for gesture classification with up to 4 and 8 missing channels respectively compared to the baseline. Notably, the proposed method is also robust to an increase in the number of missing channels compared to other methods. These results show the efficacy of using random channel ablation to improve classifier robustness for multimodal and multi-channel biosignal-based hand gesture classification.
Paper Structure (15 sections, 1 equation, 4 figures, 2 tables, 1 algorithm)

This paper contains 15 sections, 1 equation, 4 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Methods and Experimental Design
  3. Data Acquisition
  4. Model Architecture, Parameters and Metrics
  5. Data Preprocessing
  6. Robust Training for Hand Gesture Classification
  7. Evaluation
  8. Results
  9. Discussion
  10. Effect of Random Channel Ablation
  11. Effect of Number of Missing Channels
  12. A Universal All-in-One Classifier
  13. Future Research
  14. Conclusion
  15. Acknowledgments

Figures (4)

  • Figure 1: Data acquisition: (a) Hand gesture; (b) Linear ultrasound probe strapped to the forearm using a custom-designed attachment; (c) BioX FMG armband strapped to the forearm.
  • Figure 2: The model architecture to classify hand gestures from multimodal data.
  • Figure 3: Data pre-processing and ablation. (a) The $350 \times 350$-pixel forearm ultrasound image was acquired using a linear ultrasound probe. (b) 8-channel FMG data was simultaneously acquired. (c) The ultrasound image was downsized by a factor of 2. The new image dimensions were $175 \times 176$. An additional column of zeros was added to make the column width divisible by 8. (d) Ultrasound and FMG data were independently normalized and then appended per frame. The FMG data was expanded to span the rows of the ultrasound image. The dimensions of each sample were $175 \times 184$ with the last 8 columns belonging to FMG and the remaining to the downsized ultrasound image. (e) The output after random channel ablation with channels 0, 6, 8, and 15 ablated. First, the number of channels to be ablated was chosen randomly which turned out to be 4. Then, 4 channels were randomly chosen and ablated from the 16 channels.
  • Figure 4: Classification accuracy for a fixed number of channels being missing.