D-Optimality-Guided Reinforcement Learning for Efficient Open-Loop Calibration of a 3-DOF Ankle Rehabilitation Robot

TL;DR

This work tackles the challenge of accurately calibrating a multi-DOF ankle rehabilitation robot under sensing and operational constraints. It combines a Kronecker-product open-loop parameter identification framework with a simulation-guided, D-optimality-based design of experiments solved via a Proximal Policy Optimization (PPO) agent, selecting four informative postures from a candidate set of 50. The approach achieves substantial information gains, guiding robust parameter estimation with as few as four postures, and demonstrates strong cross-episode generalization in both simulation and real-world robot tests. Practically, the method reduces calibration effort and hardware wear while maintaining high alignment precision, making it broadly applicable to complex robotic rehab systems and other multi-DOF mechanisms.

Abstract

Accurate alignment of multi-degree-of-freedom rehabilitation robots is essential for safe and effective patient training. This paper proposes a two-stage calibration framework for a self-designed three-degree-of-freedom (3-DOF) ankle rehabilitation robot. First, a Kronecker-product-based open-loop calibration method is developed to cast the input-output alignment into a linear parameter identification problem, which in turn defines the associated experimental design objective through the resulting information matrix. Building on this formulation, calibration posture selection is posed as a combinatorial design-of-experiments problem guided by a D-optimality criterion, i.e., selecting a small subset of postures that maximises the determinant of the information matrix. To enable practical selection under constraints, a Proximal Policy Optimization (PPO) agent is trained in simulation to choose 4 informative postures from a candidate set of 50. Across simulation and real-robot evaluations, the learned policy consistently yields substantially more informative posture combinations than random selection: the mean determinant of the information matrix achieved by PPO is reported to be more than two orders of magnitude higher with reduced variance. In addition, real-world results indicate that a parameter vector identified from only four D-optimality-guided postures provides stronger cross-episode prediction consistency than estimates obtained from a larger but unstructured set of 50 postures. The proposed framework therefore improves calibration efficiency while maintaining robust parameter estimation, offering practical guidance for high-precision alignment of multi-DOF rehabilitation robots.

Figures (17)

• Figure 1: Formulation of a D-Optimality-Guided Posture Selection Problem.
• Figure 2: PPO-based RL Frame Workflow.
• Figure 3: End-effector attitude change relative to the world coordinate system.
• Figure 4: Distribution of input-output errors before and after open-loop calibration in simulation experiments.
• Figure 5: Experimental Procedure.