Robotic Constrained Imitation Learning for the Peg Transfer Task in Fundamentals of Laparoscopic Surgery

TL;DR

The paper addresses autonomous laparoscopic peg transfer using monocular endoscopy by introducing constrained imitation learning that derives phase-specific motion constraints from a single expert demonstration. A constrained inverse-kinematics framework maintains port-based pose constraints, while a haptically guided data-collection pipeline generates robust teaching data at 10 Hz and trains an RNNPB predictor with a parametric bias to capture variation in motion style. Experiments show that data collected under the imposed constraints yields more stable reference trajectories and higher task success compared to unconstrained data, with latent-space analyses indicating clearer differentiation of take, pass, and insert phases. The approach eliminates the need for depth images or explicit 3D models, offering a practical and transferable method for depth-ambiguous robotic tasks and potential generalization to other constrained manipulation domains.

Abstract

In this study, we present an implementation strategy for a robot that performs peg transfer tasks in Fundamentals of Laparoscopic Surgery (FLS) via imitation learning, aimed at the development of an autonomous robot for laparoscopic surgery. Robotic laparoscopic surgery presents two main challenges: (1) the need to manipulate forceps using ports established on the body surface as fulcrums, and (2) difficulty in perceiving depth information when working with a monocular camera that displays its images on a monitor. Especially, regarding issue (2), most prior research has assumed the availability of depth images or models of a target to be operated on. Therefore, in this study, we achieve more accurate imitation learning with only monocular images by extracting motion constraints from one exemplary motion of skilled operators, collecting data based on these constraints, and conducting imitation learning based on the collected data. We implemented an overall system using two Franka Emika Panda Robot Arms and validated its effectiveness.

Figures (10)

• Figure 1: The concept of this study: regarding peg transfer tasks in Fundamentals of Laparoscopic Surgery (FLS) for robots using imitation learning, we handle two main problems: (1) the forceps are constrained by laparoscopic ports and (2) only RGB information can be obtained from a monocular endoscope.
• Figure 2: The setup for robotic peg transfer tasks in Fundamentals of Laparoscopic Surgery. Franka Emika Panda Robot Arms are operated by Touch Haptic Device (3D Systems Corp.). Maryland Dissectors are controlled via Hand Adapter and Parallel Gripper to transfer Rubber Object on Peg Board.
• Figure 3: The configuration to control forceps considering the constraint by a laparoscopic port. The left figure shows a geometric model of robot arms, and the right figure shows the kinematics of forceps with a virtual linear joint from the tip of the hand that overlaps with the long axis of the forceps.
• Figure 4: Trajectory example of target forcep-tip position $z^{ref}_{forcep}$. The transition condition $S$ divides phases of the demonstration. Motion constraint $C_{i}$ and force feedback function $F_{i}$ are generated from the motion in each phase $i$.
• Figure 5: One exemplary demonstration of peg transfer. The upper figures show the human teaching with haptic devices, the middle figures show the motion of robot arms, and the lower figures show the endoscopic image. Each figure represents the image when there is a phase transition.