Learning to Score Sign Language with Two-stage Method

TL;DR

The analysis shows that choosing reconstruction tasks in the first stage can provide more expressive features, and using smoothing methods can provide an effective reference for assessment, and the method provides good score feedback mechanisms and high consistency with professional assessments compared to end-to-end evaluations.

Abstract

Human action recognition and performance assessment have been hot research topics in recent years. Recognition problems have mature solutions in the field of sign language, but past research in performance analysis has focused on competitive sports and medical training, overlooking the scoring assessment ,which is an important part of sign language teaching digitalization. In this paper, we analyze the existing technologies for performance assessment and adopt methods that perform well in human pose reconstruction tasks combined with motion rotation embedded expressions, proposing a two-stage sign language performance evaluation pipeline. Our analysis shows that choosing reconstruction tasks in the first stage can provide more expressive features, and using smoothing methods can provide an effective reference for assessment. Experiments show that our method provides good score feedback mechanisms and high consistency with professional assessments compared to end-to-end evaluations.

Figures (7)

• Figure 1: The pipeline of our system.
• Figure 2: 3D human mesh reconstruction
• Figure 3: Smoothing pose estimation results
• Figure 4: Embedding quaternion spatiotemporal sequences
• Figure 5: Align with Dtw