AI-Based Stroke Rehabilitation Domiciliary Assessment System with ST_GCN Attention

TL;DR

This work addresses the need for scalable, quantitative home-based stroke rehabilitation assessment. It introduces RAST-G@, an end-to-end system that fuses a spatio-temporal graph convolutional network with Transformer-based temporal attention to map RGB-D skeletal sequences to expert assessment scores. The authors collect the NRC upper-limb dataset and evaluate on KIMORE and NRC, showing consistent improvements in MAD, RMSE, and MAPE over baselines and providing patient-friendly feedback through a mobile app. The results demonstrate a practical path toward objective, patient-centered domiciliary rehabilitation monitoring and feedback without requiring non-patient reference data.

Abstract

Effective stroke recovery requires continuous rehabilitation integrated with daily living. To support this need, we propose a home-based rehabilitation exercise and feedback system. The system consists of (1) hardware setup with RGB-D camera and wearable sensors to capture Stroke movements, (2) a mobile application for exercise guidance, and (3) an AI server for assessment and feedback. When Stroke user exercises following the application guidance, the system records skeleton sequences, which are then Assessed by the deep learning model, RAST-G@. The model employs a spatio-temporal graph convolutional network (ST-GCN) to extract skeletal features and integrates transformer-based temporal attention to figure out action quality. For system implementation, we constructed the NRC dataset, include 10 upper-limb activities of daily living (ADL) and 5 range-of-motion (ROM) collected from stroke and non-disabled participants, with Score annotations provided by licensed physiotherapists. Results on the KIMORE and NRC datasets show that RAST-G@ improves over baseline in terms of MAD, RMSE, and MAPE. Furthermore, the system provides user feedback that combines patient-centered assessment and monitoring. The results demonstrate that the proposed system offers a scalable approach for quantitative and consistent domiciliary rehabilitation assessment.

Figures (7)

• Figure 1: Upper limb skeleton keypoints used in this Study. (a) Basic 25 configuration, (b) Additional 27 configurations.
• Figure 2: Overall System architecture. (a) Hardware setup and rehabilitation exercise interface using the proposed system (RGB-D camera, IMU, and android os mobile device), (b) Server component for storing collected data, running model inference, and managing feedback transmission.
• Figure 3: Overview of RAST-G@ Model Structure. (a) Skeleton sequence and Score input, (b) Outputs 3d feature map $X^{\prime} \in \mathbb{R}^{({N \times V}) \times T \times C}$ via STGCN Block from skeleton spatio-temporal graph. (c) Structure of a single ST-GCN unit. After GCN and TCN layers, followed by Batch Normalization and ReLU; the output is concatenated with the initial input through residual connections. (c) Transformer attention block. From the ST-GCN output, the query (Q), key (K), and value (V) are obtained, and the attention scores are computed as in (4). (d) The prediction $Y$ is compared with the ground-truth scores, $\hat{Y}$
• Figure 4: Uniform Frame Sequence Method.
• Figure 5: Visualization of NRC works(yellow) compare to theraphists' score(blue) during 4 month long-term dataset. label02: brushing hair, Stroke 14.