Breast-Rehab: A Postoperative Breast Cancer Rehabilitation Training Assessment System Based on Human Action Recognition
Zikang Chen, Tan Xie, Qinchuan Wang, Heming Zheng, Xudong Lu
TL;DR
This paper tackles postoperative upper-limb rehabilitation for breast cancer patients by integrating a skeleton-guided action recognition pipeline with a retrieval-augmented generation framework to segment home-video exercises and produce clinically grounded assessment reports. It introduces a dual-stream architecture that fuses visual and 3D skeletal data, enhanced by motion prompts and cross-attention, and leverages a 1M-chunk Breast-CRAG knowledge base for RAG-based reporting. The approach achieves superior action recognition accuracy (0.72) compared with mm-LLMs and CLIP-AR, and significantly improves report quality over baselines, as shown in a two-week clinical pilot with 15 participants. The study demonstrates a low-cost, scalable solution for remote rehabilitation monitoring, reducing computational overhead and hallucinations while enabling real-time clinician feedback and patient guidance, with plans for a larger randomized trial to validate efficacy.
Abstract
Postoperative upper limb dysfunction is prevalent among breast cancer survivors, yet their adherence to at-home rehabilitation exercises is low amidst limited nursing resources. The hardware overhead of commonly adopted VR-based mHealth solutions further hinders their widespread clinical application. Therefore, we developed Breast-Rehab, a novel, low-cost mHealth system to provide patients with out-of-hospital upper limb rehabilitation management. Breast-Rehab integrates a bespoke human action recognition algorithm with a retrieval-augmented generation (RAG) framework. By fusing visual and 3D skeletal data, our model accurately segments exercise videos recorded in uncontrolled home environments, outperforming standard models. These segmented clips, combined with a domain-specific knowledge base, guide a multi-modal large language model to generate clinically relevant assessment reports. This approach significantly reduces computational overhead and mitigates model hallucinations. We implemented the system as a WeChat Mini Program and a nurse-facing dashboard. A preliminary clinical study validated the system's feasibility and user acceptance, with patients achieving an average exercise frequency of 0.59 sessions/day over a two-week period. This work thus presents a complete, validated pipeline for AI-driven, at-home rehabilitation monitoring.