Hysteresis Compensation of Flexible Continuum Manipulator using RGBD Sensing and Temporal Convolutional Network

TL;DR

The paper tackles hysteresis in long, cable-driven flexible continuum manipulators used for endoscopic surgery by leveraging RGBD-based fiducial marker pose estimation and data-driven learning. It compares multiple deep learning architectures and finds a Temporal Convolution Network (TCN) with sequence length $L=80$ to be most effective for both forward and inverse hysteresis modeling, enabling a two-stage compensation algorithm. The authors implement this as a calibrated controller that refines desired joint commands to offset hysteresis, achieving substantial reductions in tracking errors on unseen trajectories—13.70 mm down to 5.29 mm in position and 31.17° down to 11.21° in orientation. This approach promises improved precision in surgical tasks, though data collection time (~256 minutes) and cable slack pose practical challenges for real-world deployment.

Abstract

Flexible continuum manipulators are valued for minimally invasive surgery, offering access to confined spaces through nonlinear paths. However, cable-driven manipulators face control difficulties due to hysteresis from cabling effects such as friction, elongation, and coupling. These effects are difficult to model due to nonlinearity and the difficulties become even more evident when dealing with long and coupled, multi-segmented manipulator. This paper proposes a data-driven approach based on Deep Neural Networks (DNN) to capture these nonlinear and previous states-dependent characteristics of cable actuation. We collect physical joint configurations according to command joint configurations using RGBD sensing and 7 fiducial markers to model the hysteresis of the proposed manipulator. Result on a study comparing the estimation performance of four DNN models show that the Temporal Convolution Network (TCN) demonstrates the highest predictive capability. Leveraging trained TCNs, we build a control algorithm to compensate for hysteresis. Tracking tests in task space using unseen trajectories show that the proposed control algorithm reduces the average position and orientation error by 61.39% (from 13.7mm to 5.29 mm) and 64.04% (from 31.17° to 11.21°), respectively. This result implies that the proposed calibrated controller effectively reaches the desired configurations by estimating the hysteresis of the manipulator. Applying this method in real surgical scenarios has the potential to enhance control precision and improve surgical performance.

Figures (11)

• Figure 1: The 3D printed fiducial markers are attached to the forceps to estimate the physical joint angles of the proposed continuum manipulator. We use a RGBD camera to detect the central position of fiducial marker.
• Figure 2: Design Parameters of Flexure Hinge Module: The design of the flexure hinges in the manipulator is based on the Circular Flexure Hinge Design by Paros and Weisbord Tseytlin2002NotchFH.
• Figure 3: Manipulator Components, Coordinate Systems, and Geometric Relationships (a) The proposed manipulator is constructed by connecting segment 1, segment 2 and forceps. (b) Coordinates attached to a single module (c) Geometric depiction of the flexure hinge module.
• Figure 4: DOFs Configuration and Cable Relationships in The Proposed Continuum Manipulator: (a) Description of the individual DOFs ($q_1$, $q_2$, $q_3$, $q_4$, and $q_5$) for manipulator. Specifically, segment 1(elbow) has 2 DOFs for pitch and yaw direction bending, driven by cables $w_{1-4}$, denoted as $q_1$ and $q_2$, segment 2(wrist) has 1 DOFs for pitch direction bending, driven by cables $w_{5, \: 6}$, represented as $q_3$ and the gripper(forceps) is equipped with 2 DOFs, driven by cables $w_{7-10}$, for left forceps angle ($q_4$) and right forceps angle ($q_5$), (b) Geometric representation of cable variations $w_i$ during the bending of adjacent hinge (Magnified view of the red-boxed region showing an ideally bent segment 2).
• Figure 5: Schematic Illustration of Manipulators with Fiducial Markers and RGBD Sensing: The proposed manipulator with $\mathrm{4.8 mm}$ in diameter and $\mathrm{2.5 m}$ in length. Seven fiducial markers are attached to capture the physical pose of the manipulator.