See, Learn, Assist: Safe and Self-Paced Robotic Rehabilitation via Video-Based Learning from Demonstration

Abstract

In this paper, we propose a novel framework that allows therapists to teach robot-assisted rehabilitation exercises remotely via RGB-D video. Our system encodes demonstrations as 6-DoF body-centric trajectories using Cartesian Dynamic Movement Primitives (DMPs), ensuring accurate posture-independent spatial generalization across diverse patient anatomies. Crucially, we execute these trajectories through a decoupled hybrid control architecture that constructs a spatially compliant virtual tunnel, paired with an effort-based temporal dilation mechanism. This architecture is applied to three distinct rehabilitation modalities: Passive, Active-Assisted, and Active-Resistive, by dynamically linking the exercise's execution phase to the patient's tangential force contribution. To guarantee safety, a Gaussian Mixture Regression (GMR) model is learned on-the-fly from the patient's own limb. This allows the detection of abnormal interaction forces and, if necessary, reverses the trajectory to prevent injury. Experimental validation demonstrates the system's precision, achieving an average trajectory reproduction error of 3.7cm and a range of motion (ROM) error of 5.5 degrees. Furthermore, dynamic interaction trials confirm that the controller successfully enforces effort-based progression while maintaining strict spatial path adherence against human disturbances.

Figures (6)

• Figure 1: Hardware architecture of the system (left) and body-centric frames constructed from skeleton keypoints (right).
• Figure 2: A GMR model learned from a subject's limb during elbow flexion-extension exercise (left). The safety corridor is defined as a $\pm5$ standard deviation around the mean expected force (in red). (Right) Exercise trajectories during shoulder abduction-adduction executed at varying trunk rotation angles. The trajectories are aligned at their starting points to demonstrate the high fidelity of the body-centric encoding.
• Figure 3: Comparison of time-normalized joint angle trajectories for the three upper-limb exercises. Orange lines show the demonstration (subject 1), while green and purple lines show the robot reproductions for subjects 1 and 2.
• Figure 4: Validation of the virtual tunnel and energy-based temporal dilation. (Left) The subject intentionally applies significant orthogonal forces (in red) while varying tangential effort (in blue). Exercise progress (green) is driven only by $f_t$, slowing when $f_t < 0$. (Right) 3D Cartesian paths confirm that the stiff orthogonal admittance maintains spatial adherence to the reference trajectory despite the significant force applied.
• Figure 5: Exercise progress vs patient's delivered force in the active-assisted and active-resistive modes.