Neural Collision Detection for Multi-arm Laparoscopy Surgical Robots Through Learning-from-Simulation

TL;DR

The paper presents an integrated framework for collision detection in multi-arm laparoscopic robots by marrying an analytical Bézier-based proximity model, a Unity simulation platform, and a neural predictor trained on a large synthetic dataset. The system uses $d_{min}$ estimation to issue real-time collision warnings with a threshold near $0.2~\mathrm{m}$, validated through both simulation and physical experiments on Kinova Gen3 arms. Key contributions include a robust analytical benchmark, a scalable simulation-based data generator totaling $75{,}655$ configurations, and a neural regressor achieving $\text{MAE}=282.2$ mm and $R^2=0.85$ on held-out data. The results demonstrate strong agreement with ground-truth measurements and highlight the potential of learning-from-simulation to enhance safety and reliability in complex robotic surgery environments.

Abstract

This study presents an integrated framework for enhancing the safety and operational efficiency of robotic arms in laparoscopic surgery by addressing key challenges in collision detection and minimum distance estimation. By combining analytical modeling, real-time simulation, and machine learning, the framework offers a robust solution for ensuring safe robotic operations. An analytical model was developed to estimate the minimum distances between robotic arms based on their joint configurations, offering precise theoretical calculations that serve as both a validation tool and a benchmark. To complement this, a 3D simulation environment was created to model two 7-DOF Kinova robotic arms, generating a diverse dataset of configurations for collision detection and distance estimation. Using these insights, a deep neural network model was trained with joint actuators of robot arms and relative positions as inputs, achieving a mean absolute error of 282.2 mm and an R-squared value of 0.85. The close alignment between predicted and actual distances highlights the network's accuracy and its ability to generalize spatial relationships. This work demonstrates the effectiveness of combining analytical precision with machine learning algorithms to enhance the precision and reliability of robotic systems.

Figures (11)

• Figure 1: (a) Robot-controlled laparoscopic instruments with an integrated laparoscope camera providing a magnified operative view. (b) A surgeon operating through a bimanual haptic interface within a multi-robot robotic laparoscopic system (RLS), with real-time visualization displayed on the monitor and overhead surgical display. These images illustrate the typical setup of a robot-assisted laparoscopic surgery environment, demonstrating both the operative field and the surgeon console. Images courtesy of CMR Surgical (Versius Surgical System).
• Figure 2: Overview of the proposed learning from simulation collision detection framework. The Unity-based simulation generates 75,655 robot configurations for training, using joint configurations and relative poses as inputs. A deep NN predicts the minimum distance ($d_{min}$) between the arms during inference on real robotic setups, issuing an audio warning when $d_{min} < 0.2~\text{m}$.
• Figure 3: Overview of the experimental setup used in this research. The configuration includes: (left) a bimanual haptic surgeon interface for controlling both robotic arms with left and right hands and a foot pedal for robot selection; (center) a multibody simulator that integrates robot kinematics, collision detection, and redundancy resolution; and (right) dual Kinova Gen3 collaborative robotic arms performing coordinated manipulation tasks.
• Figure 4: Modeling of the Kinova Gen3 7-DOF robotic arm. (a) Rotation of joint angles illustrating the seven revolute joints ($q_1$ to $q_7$) that define the manipulator’s full configuration and contribute to the positioning and orientation of the end effector. (b) Denavit-Hartenberg (DH) frame definitions showing coordinate frame assignments for each joint according to the DH convention, representing the spatial orientation and relative displacement between adjacent links.
• Figure 5: Simulation of two Kinova Gen3 robotic arms in Unity 3D. The environment models the dual-arm setup with accurate link geometries, actuator hierarchies, and kinematic constraints, enabling realistic motion simulation, collision visualization, and data generation for learning-based analysis.