Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

XiCAD: Camera Activation Detection in the Da Vinci Xi User Interface

Alexander C. Jenke, Gregor Just, Claas de Boer, Martin Wagner, Sebastian Bodenstedt, Stefanie Speidel

TL;DR

A lightweight pipeline based on a ResNet18 convolutional neural network enables reliable, real-time extraction of camera activation metadata from surgical videos, facilitating automated preprocessing and analysis for diverse downstream applications.

Abstract

Purpose: Robot-assisted minimally invasive surgery relies on endoscopic video as the sole intraoperative visual feedback. The DaVinci Xi system overlays a graphical user interface (UI) that indicates the state of each robotic arm, including the activation of the endoscope arm. Detecting this activation provides valuable metadata such as camera movement information, which can support downstream surgical data science tasks including tool tracking, skill assessment, or camera control automation. Methods: We developed a lightweight pipeline based on a ResNet18 convolutional neural network to automatically identify the position of the camera tile and its activation state within the DaVinci Xi UI. The model was fine-tuned on manually annotated data from the SurgToolLoc dataset and evaluated across three public datasets comprising over 70,000 frames. Results: The model achieved F1-scores between 0.993 and 1.000 for the binary detection of active cameras and correctly localized the camera tile in all cases without false multiple-camera detections. Conclusion: The proposed pipeline enables reliable, real-time extraction of camera activation metadata from surgical videos, facilitating automated preprocessing and analysis for diverse downstream applications. All code, trained models, and annotations are publicly available.

XiCAD: Camera Activation Detection in the Da Vinci Xi User Interface

TL;DR

A lightweight pipeline based on a ResNet18 convolutional neural network enables reliable, real-time extraction of camera activation metadata from surgical videos, facilitating automated preprocessing and analysis for diverse downstream applications.

Abstract

Purpose: Robot-assisted minimally invasive surgery relies on endoscopic video as the sole intraoperative visual feedback. The DaVinci Xi system overlays a graphical user interface (UI) that indicates the state of each robotic arm, including the activation of the endoscope arm. Detecting this activation provides valuable metadata such as camera movement information, which can support downstream surgical data science tasks including tool tracking, skill assessment, or camera control automation. Methods: We developed a lightweight pipeline based on a ResNet18 convolutional neural network to automatically identify the position of the camera tile and its activation state within the DaVinci Xi UI. The model was fine-tuned on manually annotated data from the SurgToolLoc dataset and evaluated across three public datasets comprising over 70,000 frames. Results: The model achieved F1-scores between 0.993 and 1.000 for the binary detection of active cameras and correctly localized the camera tile in all cases without false multiple-camera detections. Conclusion: The proposed pipeline enables reliable, real-time extraction of camera activation metadata from surgical videos, facilitating automated preprocessing and analysis for diverse downstream applications. All code, trained models, and annotations are publicly available.

Paper Structure

This paper contains 9 sections, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Method
  3. DaVinci Xi UI
  4. Neural network
  5. Datasets
  6. Experiment Setup
  7. Results
  8. Discussion
  9. Conclusion

Figures (3)

  • Figure 1: Overview of the proposed pipeline for detecting the camera tile and its activation state in the DaVinci Xi user interface. Four tiles are cropped out of the frame, each is classified by a fine-tuned ResNet18 as no camera, inactive camera, or active camera. The tile-level predictions are then combined through simple logic to yield the final frame-level camera activation and position.
  • Figure 2: Macro-averaged F1-score across all three tile classes (no camera, inactive camera, active camera) during training over 100 epochs, showing convergence and achievement of perfect classification.
  • Figure 3: Confusion matrices for tile-level (a) and frame-level (b–d) predictions on the test datasets. Absolute counts are shown, with color indicating percentage normalized over true labels.