Accounting for Hysteresis in the Forward Kinematics of Nonlinearly-Routed Tendon-Driven Continuum Robots via a Learned Deep Decoder Network

TL;DR

This work tackles the problem of hysteresis in forward kinematics for tendon-driven continuum robots by introducing a deep decoder that directly outputs the robot's full 3D shape as a point cloud, conditioned on both current and prior tendon configurations. The method integrates a novel loss combining Chamfer distance and Earth Mover's Distance to produce dense, well-distributed point clouds and demonstrates significant improvements over physics-based models and non-hysteresis learners. Data is collected with dual RGB-D sensing and a carefully constructed hysteresis dataset, enabling robust learning of history-dependent shapes. Experiments show the approach achieves faster, more accurate full-shape predictions and effectively accounts for hysteresis, which has practical implications for planning and control in complex medical environments.

Abstract

Tendon-driven continuum robots have been gaining popularity in medical applications due to their ability to curve around complex anatomical structures, potentially reducing the invasiveness of surgery. However, accurate modeling is required to plan and control the movements of these flexible robots. Physics-based models have limitations due to unmodeled effects, leading to mismatches between model prediction and actual robot shape. Recently proposed learning-based methods have been shown to overcome some of these limitations but do not account for hysteresis, a significant source of error for these robots. To overcome these challenges, we propose a novel deep decoder neural network that predicts the complete shape of tendon-driven robots using point clouds as the shape representation, conditioned on prior configurations to account for hysteresis. We evaluate our method on a physical tendon-driven robot and show that our network model accurately predicts the robot's shape, significantly outperforming a state-of-the-art physics-based model and a learning-based model that does not account for hysteresis.

Figures (9)

• Figure 1: (Upper left) We leverage point clouds to represent the whole shape of the robot. (Upper right) Hysteresis causes the robot's shape to significantly depend on its prior configuration history. History $1$ and History $2$ show the sensed shape of the robot at the same tendon-displacement configuration, but having come from different prior configurations. (Bottom) Our proposed deep decoder neural network model aims to learn the tendon robot's entire shape while accounting for hysteresis. Taking as input the current and previous tendon configurations it produces a point cloud (white) that well aligns with the ground truth (red).
• Figure 2: Network architecture of our novel deep decoder network. The model takes the robot's augmented configuration vector $\boldsymbol{\eta}$ as input and outputs a point cloud of the robot's shape $\hat{\mathbf{p}}$. The model consists of $4$ hidden layers, each of which is fully-connected, ReLU activated, and batch normalized followed by an output layer of size $3M$. We reshape the network output (1-dimensional vector) into a $3 \times M$ matrix, where $M$ is the number of points, as the point cloud representation.
• Figure 3: Registration and Segmentation. (Left) We sense point clouds of the robot's shape using two RGB-D cameras placed around the robot. (Middle) We apply a registration algorithm to align the two camera frames with the robot base frame. The concatenated, aligned point clouds are shown. (Right) We then segment the robot's point cloud by removing point cloud points outside of the robot's workspace.
• Figure 4: Qualitative loss function comparison. The sensed, ground truth robot shape point cloud is shown in red, and the model-predicted point clouds are shown in blue. The model trained with MSE loss completely fails to achieve the robot's shape. The model trained with the Chamfer loss function produces point clouds that show the robot's geometry but the resulting points are unevenly distributed along the shape. Our proposed loss function, EMD + Chamfer ($\mathcal{L}_{tendon}$), shows the best performance both qualitatively and quantitatively, demonstrating a precise estimation of the robot's shape with evenly distributed points.
• Figure 5: Physics-based predicted robot shape (blue) vs. ground truth (red). Three examples are shown from the test set, the worst (left), the median (middle), and the best (right). These demonstrate the ranges of error between the predicted shape and the ground truth sensed shape.