Skeleton-Based Transformer for Classification of Errors and Better Feedback in Low Back Pain Physical Rehabilitation Exercises

TL;DR

This work addresses the need for detailed automated feedback in rehabilitation by introducing a skeleton-based Transformer that classifies errors in specific exercises. Built on a Hyperformer-inspired architecture, it partitions joints into hypergraphs and employs self-attention to capture spatial-temporal relations, training a separate model for each exercise. On the KERAAL dataset, the approach achieves state-of-the-art accuracy over baselines and provides biomechanically meaningful joint-importance insights to guide patient feedback. The method shows promise for in-home rehabilitation support, with future directions including data augmentation and temporal localization of errors to further enhance feedback quality.

Abstract

Physical rehabilitation exercises suggested by healthcare professionals can help recovery from various musculoskeletal disorders and prevent re-injury. However, patients' engagement tends to decrease over time without direct supervision, which is why there is a need for an automated monitoring system. In recent years, there has been great progress in quality assessment of physical rehabilitation exercises. Most of them only provide a binary classification if the performance is correct or incorrect, and a few provide a continuous score. This information is not sufficient for patients to improve their performance. In this work, we propose an algorithm for error classification of rehabilitation exercises, thus making the first step toward more detailed feedback to patients. We focus on skeleton-based exercise assessment, which utilizes human pose estimation to evaluate motion. Inspired by recent algorithms for quality assessment during rehabilitation exercises, we propose a Transformer-based model for the described classification. Our model is inspired by the HyperFormer method for human action recognition, and adapted to our problem and dataset. The evaluation is done on the KERAAL dataset, as it is the only medical dataset with clear error labels for the exercises, and our model significantly surpasses state-of-the-art methods. Furthermore, we bridge the gap towards better feedback to the patients by presenting a way to calculate the importance of joints for each exercise.

Figures (8)

• Figure 1: The three rehabilitation exercises in the Keraal dataset. Image sourced from Nguyen2024IJCNN.
• Figure 2: Errors descriptions for all three exercises in the Keraal dataset. Image sourced from Nguyen2024IJCNN.
• Figure 3: Groups of skeletal joints (hypergraphs) on the left and model overview on the right. Right part taken from hyperformer2023
• Figure 4: Overview of the hyper self-attention module hyperformer2023.
• Figure 5: Confusion matrices for each of 3 exercises for the first case (trained on group3 and tested on groups 2A and 1A ). Rows full of 0.00 mean that the corresponding error is not present in the test set.