Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multimodal Reaching-Position Prediction for ADL Support Using Neural Networks

Yutaka Takase, Kimitoshi Yamazaki

TL;DR

A reaching-position prediction scheme is proposed that targets the motion of lifting the upper arm, which is burdensome for patients with hemiplegia and the elderly in daily living activities and achieves accuracy of 93% macro average and F1-score of 0.69 for a 9-class classification prediction at 35% of the motion completion.

Abstract

This study aimed to develop daily living support robots for patients with hemiplegia and the elderly. To support the daily living activities using robots in ordinary households without imposing physical and mental burdens on users, the system must detect the actions of the user and move appropriately according to their motions. We propose a reaching-position prediction scheme that targets the motion of lifting the upper arm, which is burdensome for patients with hemiplegia and the elderly in daily living activities. For this motion, it is difficult to obtain effective features to create a prediction model in environments where large-scale sensor system installation is not feasible and the motion time is short. We performed motion-collection experiments, revealed the features of the target motion and built a prediction model using the multimodal motion features and deep learning. The proposed model achieved an accuracy of 93 \% macro average and F1-score of 0.69 for a 9-class classification prediction at 35\% of the motion completion.

Multimodal Reaching-Position Prediction for ADL Support Using Neural Networks

TL;DR

A reaching-position prediction scheme is proposed that targets the motion of lifting the upper arm, which is burdensome for patients with hemiplegia and the elderly in daily living activities and achieves accuracy of 93% macro average and F1-score of 0.69 for a 9-class classification prediction at 35% of the motion completion.

Abstract

This study aimed to develop daily living support robots for patients with hemiplegia and the elderly. To support the daily living activities using robots in ordinary households without imposing physical and mental burdens on users, the system must detect the actions of the user and move appropriately according to their motions. We propose a reaching-position prediction scheme that targets the motion of lifting the upper arm, which is burdensome for patients with hemiplegia and the elderly in daily living activities. For this motion, it is difficult to obtain effective features to create a prediction model in environments where large-scale sensor system installation is not feasible and the motion time is short. We performed motion-collection experiments, revealed the features of the target motion and built a prediction model using the multimodal motion features and deep learning. The proposed model achieved an accuracy of 93 \% macro average and F1-score of 0.69 for a 9-class classification prediction at 35\% of the motion completion.
Paper Structure (26 sections, 5 figures, 3 tables)

This paper contains 26 sections, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Research questions and Our approaches
  4. Research questions
  5. Approach
  6. Analysis of reaching motions
  7. Motion collection
  8. Motion analysis
  9. Modals for prediction
  10. Motion data extraction
  11. Prediction Model
  12. Features
  13. Face Features
  14. Depth Features
  15. Motion Features
  16. ...and 11 more sections

Figures (5)

  • Figure 1: Overview of data collection environment
  • Figure 2: Visualization of variations within 10 frames after the start of the motion using the sum of absolute difference (SAD). A, B: Examples from the collected motions. C : Result based on the entire collected motions.
  • Figure 3: Example of the collected motion data; it is a sequence of color images subject to reach the center-left region. The number indicates elapsed frames from the start of the motion.
  • Figure 4: Multimodal late fusion model
  • Figure 5: Confusion matrix of the proposed fusion model. the percentages represent classification rate.The asterisks indicate pairs for which a significant difference in motion speed was found as a result of the post-hoc Tukey HSD test describe in Section \ref{['sec:motion_analysis']}.