A benchmark for video-based laparoscopic skill analysis and assessment

TL;DR

The Laparoscopic Skill Analysis and Assessment (LASANA) dataset is introduced, comprising 1270 stereo video recordings of four basic laparoscopic training tasks, annotated with a structured skill rating, aggregated from three independent raters, as well as binary labels indicating the presence or absence of task-specific errors.

Abstract

Laparoscopic surgery is a complex surgical technique that requires extensive training. Recent advances in deep learning have shown promise in supporting this training by enabling automatic video-based assessment of surgical skills. However, the development and evaluation of deep learning models is currently hindered by the limited size of available annotated datasets. To address this gap, we introduce the Laparoscopic Skill Analysis and Assessment (LASANA) dataset, comprising 1270 stereo video recordings of four basic laparoscopic training tasks. Each recording is annotated with a structured skill rating, aggregated from three independent raters, as well as binary labels indicating the presence or absence of task-specific errors. The majority of recordings originate from a laparoscopic training course, thereby reflecting a natural variation in the skill of participants. To facilitate benchmarking of both existing and novel approaches for video-based skill assessment and error recognition, we provide predefined data splits for each task. Furthermore, we present baseline results from a deep learning model as a reference point for future comparisons.

Figures (12)

• Figure 1: Experimental setup for recording laparoscopic training tasks. The setup includes (A) a laparoscopic training box containing task-specific objects and materials, (B) a stereo endoscope, (C) a 2D monitor displaying the live video feed from the endoscope's left camera, and (D) standard laparoscopic instruments.
• Figure 2: The recorded laparoscopic training tasks.
• Figure 3: Timeline of video recording during the laparoscopic training course. Filled circles indicate course lessons, while open circles indicate recording sessions.
• Figure 4: Total GRS for suture & knot recordings assigned by $r_1$ and $r_3$ before (left) and after (right) normalization.
• Figure 5: Distributions of the total global rating scores per task in the LASANA dataset.