AutoCam: Hierarchical Path Planning for an Autonomous Auxiliary Camera in Surgical Robotics

TL;DR

AutoCam addresses the need for an autonomous auxiliary camera in RAMIS to enhance visualization without adding surgeon workload. It introduces a hierarchical controller that first computes a naive geometric pose and then refines it with a constrained inverse kinematics solver under workspace and joint-limit constraints, enabling full $6$-DOF camera motion. Markerless arm calibration, a No-Go Zone constraint framework, and a robust optimization with Huber loss support safe, real-time operation on the dVRK, with an open-source implementation. Experimental results show high visibility ($99.84\%$) and precise tracking (e.g., $4.36 \pm 2.11$ degrees orientation error and $1.95 \pm 5.66$ mm distance error) and fast loop times ($6.8 \pm 12.8$ ms), indicating AutoCam's potential to enable multi-camera visualization in RAMIS and assist novice training. These findings suggest a practical path toward semi-autonomous camera control that reduces occlusions and improves spatial understanding during robotic surgery.

Abstract

Incorporating an autonomous auxiliary camera into robot-assisted minimally invasive surgery (RAMIS) enhances spatial awareness and eliminates manual viewpoint control. Existing path planning methods for auxiliary cameras track two-dimensional surgical features but do not simultaneously account for camera orientation, workspace constraints, and robot joint limits. This study presents AutoCam: an automatic auxiliary camera placement method to improve visualization in RAMIS. Implemented on the da Vinci Research Kit, the system uses a priority-based, workspace-constrained control algorithm that combines heuristic geometric placement with nonlinear optimization to ensure robust camera tracking. A user study (N=6) demonstrated that the system maintained 99.84% visibility of a salient feature and achieved a pose error of 4.36 $\pm$ 2.11 degrees and 1.95 $\pm$ 5.66 mm. The controller was computationally efficient, with a loop time of 6.8 $\pm$ 12.8 ms. An additional pilot study (N=6), where novices completed a Fundamentals of Laparoscopic Surgery training task, suggests that users can teleoperate just as effectively from AutoCam's viewpoint as from the endoscope's while still benefiting from AutoCam's improved visual coverage of the scene. These results indicate that an auxiliary camera can be autonomously controlled using the da Vinci patient-side manipulators to track a salient feature, laying the groundwork for new multi-camera visualization methods in RAMIS.

Figures (9)

• Figure 1: AutoCam path planning framework.
• Figure 2: a) Homogeneous transformations and vectors for geometric-based camera placement. Transformations ${}^{ECM}\boldsymbol{T}_{cam}^d$ and ${}^{ECM}\boldsymbol{T}_{feature}$ map the endoscopic camera frame to the autonomous camera and salient feature frames, respectively. The vector $\boldsymbol{n}$ is normal to the tracked feature, and $d_{t}$ is the desired viewing distance between the feature and the camera. The start and end of the single-handed wire-chaser task are shown; b) The da Vinci® ProGrasp Forceps holding the "Stable Grasp" mount to control the auxiliary camera; c) The No-Go Zone, illustrating a plane's normal vector $\boldsymbol{n}_{plane}$ and point $\boldsymbol{p}_{plane}$ on the plane. Also shown in the projection of the desired position $\boldsymbol{p}_{cam}$ onto the boundary as $\boldsymbol{p}_{boundary}$.
• Figure 3: Top: The user interface (UI) and the console view with vertically concatenated ECM and autonomous camera frames. The UI allows adjustment of the on-screen separation (disparity) of the left/right autonomous camera frames. Bottom: Network diagram of all data streams. PC1 receives end-effector poses via the dVRK API, and sends commands over a FireWire connection to move the PSMs. The autonomous camera is interfaced by two Raspberry Pi® controllers and video is sent to PC2 over ethernet for processing. The ECM video is captured and relayed to PC2 via a DeckLink capture card. Concatenated ECM and autonomous camera frames are sent to the left/right console viewports via an HDMI cable.
• Figure 4: Absolute tracking errors (black) with bars (red, green, blue, organge, cyan) indicating when workspace and robot constraints were encountered. a) Angular distance between the auxiliary camera's optical axis and the vector connecting the its origin to the feature origin ($VVA$); b) Error between desired and actual camera-to-feature distance ($FD$); c) Angular error between camera and world horizontal axes ($PF$).
• Figure 5: a) and b) Orientation and position error ($AN$ and $PN$) between the naive pose computed by the geometric-based placement and the actual pose from the hierarchical controller. Horizontal bars indicate constraint conditions; c) Example of the naive (green), actual (red), and feature trajectories (blue) with workspace constraints (grey) shown.