Hybrid Visual Servoing of Tendon-driven Continuum Robots

TL;DR

This work tackles tendon-driven continuum robot control in dynamic, unstructured environments by marrying image-based and deep learning visual servoing. The proposed Hybrid Visual Servoing (HVS) framework switches between IBVS and DLBVS via a SAD threshold and feature-visibility checks, combining IBVS’s fast, accurate convergence with DLBVS’s robustness to occlusions and lighting changes. Key contributions include a detailed IBVS formulation for TDCRs, a CNN-based DLBVS with transfer learning on a VGG-16 backbone, a 5000-image Blender dataset with occlusions, and a rigorous evaluation showing faster convergence, lower final error, and smoother trajectories under disturbances. The results demonstrate that HVS outperforms DLBVS alone while preserving robustness, offering a practical approach for reliable TDCR operation in real-world, unstructured scenes.

Abstract

This paper introduces a novel Hybrid Visual Servoing (HVS) approach for controlling tendon-driven continuum robots (TDCRs). The HVS system combines Image-Based Visual Servoing (IBVS) with Deep Learning-Based Visual Servoing (DLBVS) to overcome the limitations of each method and improve overall performance. IBVS offers higher accuracy and faster convergence in feature-rich environments, while DLBVS enhances robustness against disturbances and offers a larger workspace. By enabling smooth transitions between IBVS and DLBVS, the proposed HVS ensures effective control in dynamic, unstructured environments. The effectiveness of this approach is validated through simulations and real-world experiments, demonstrating that HVS achieves reduced iteration time, faster convergence, lower final error, and smoother performance compared to DLBVS alone, while maintaining DLBVS's robustness in challenging conditions such as occlusions, lighting changes, actuator noise, and physical impacts.

Figures (9)

• Figure 1: Block diagram of the deep learning-based visual servoing controller.
• Figure 2: The test setup consists of the TDCR with a camera attached to its tip.
• Figure 3: The sequence of camera views in the simulation began with tendon displacements of $(q_1, q_2) = (-10, 9)$ mm.
• Figure 4: The simulation results in Blender software demonstrate the performance of the HVS controller.
• Figure 5: Camera views under normal conditions with initial tendon displacements of $(10, 8)$, $(10, -8)$, $(-10, 8)$, and $(-10, -8)$.