Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evaluation Framework for Feedback Generation Methods in Skeletal Movement Assessment

Tal Hakim

TL;DR

The paper tackles the gap in automatic movement assessment by focusing on feedback generation, not just scoring, for skeletal movement analysis. It proposes a formal terminology and evaluation criteria to classify and compare feedback-generation methods, distinguishing bound versus unbound feedback, temporal granularity, online versus offline modes, and modalities. It introduces evaluation metrics such as temporal IoU and feature IoU to assess feedback accuracy and discusses annotation challenges for feedback ground-truth. As the first work to formulate feedback generation in skeletal movement assessment, it aims to standardize comparisons and accelerate the development of interpretable, actionable feedback for at-home rehabilitation.

Abstract

The application of machine-learning solutions to movement assessment from skeleton videos has attracted significant research attention in recent years. This advancement has made rehabilitation at home more accessible, utilizing movement assessment algorithms that can operate on affordable equipment for human pose detection and analysis from 2D or 3D videos. While the primary objective of automatic assessment tasks is to score movements, the automatic generation of feedback highlighting key movement issues has the potential to significantly enhance and accelerate the rehabilitation process. While numerous research works exist in the field of automatic movement assessment, only a handful address feedback generation. In this study, we propose terminology and criteria for the classification, evaluation, and comparison of feedback generation solutions. We discuss the challenges associated with each feedback generation approach and use our proposed criteria to classify existing solutions. To our knowledge, this is the first work that formulates feedback generation in skeletal movement assessment.

Evaluation Framework for Feedback Generation Methods in Skeletal Movement Assessment

TL;DR

The paper tackles the gap in automatic movement assessment by focusing on feedback generation, not just scoring, for skeletal movement analysis. It proposes a formal terminology and evaluation criteria to classify and compare feedback-generation methods, distinguishing bound versus unbound feedback, temporal granularity, online versus offline modes, and modalities. It introduces evaluation metrics such as temporal IoU and feature IoU to assess feedback accuracy and discusses annotation challenges for feedback ground-truth. As the first work to formulate feedback generation in skeletal movement assessment, it aims to standardize comparisons and accelerate the development of interpretable, actionable feedback for at-home rehabilitation.

Abstract

The application of machine-learning solutions to movement assessment from skeleton videos has attracted significant research attention in recent years. This advancement has made rehabilitation at home more accessible, utilizing movement assessment algorithms that can operate on affordable equipment for human pose detection and analysis from 2D or 3D videos. While the primary objective of automatic assessment tasks is to score movements, the automatic generation of feedback highlighting key movement issues has the potential to significantly enhance and accelerate the rehabilitation process. While numerous research works exist in the field of automatic movement assessment, only a handful address feedback generation. In this study, we propose terminology and criteria for the classification, evaluation, and comparison of feedback generation solutions. We discuss the challenges associated with each feedback generation approach and use our proposed criteria to classify existing solutions. To our knowledge, this is the first work that formulates feedback generation in skeletal movement assessment.
Paper Structure (15 sections, 1 equation, 3 figures, 1 table)

This paper contains 15 sections, 1 equation, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Movement Assessment
  3. Feedback Generation
  4. Scoring Model
  5. Feedback Boundness and Order
  6. Temporal Granularity
  7. Offline/Online Feedback Generation Mode
  8. Feedback Modality
  9. Evaluating Generated Feedback
  10. Bound Feedback Labels
  11. Unbound Feedback Labels
  12. Feature IoU
  13. Online-Feedback Evaluation
  14. Discussion
  15. Conclusions

Figures (3)

  • Figure 1: Examples of raw feature deviations. A green, clear cell indicates a proper, in-range feature value, while cells highlighted in red indicate deviating feature values. Each deviation can be directly translated into a first-order feedback label. When a feature consistently deviates over a sequence of frames, temporal aggregation will consolidate the sequence of first-order feedback labels into a single feedback label. Similarly, when related features deviate simultaneously, skeletal aggregation will consolidate the set of related first-order feedback labels into a single feedback label.
  • Figure 2: Different approaches to adding temporal notation to bound feedback labels: (a) different feedback labels are detected along the movement time axis using sliding windows of different lengths; (b) a set of time frames is defined to support temporally local feedback generation; and (c) temporally local feedback labels are achieved using a Cartesian product of the sets of feedback labels and time frames.
  • Figure 3: Visualizing multiple first-order feedback labels in (a), versus a single second-order feedback label in (b). This example demonstrates the power of visual feedback over textual feedback: the three reported deviations in (a) are first-order, yet, they provide intuitive feedback, leaving negligible added value for the aggregated feedback label in (b). This approach has the potential to enable unbound feedback generation without addressing the skeletal aggregation problem.