Cataract-LMM: Large-Scale, Multi-Source, Multi-Task Benchmark for Deep Learning in Surgical Video Analysis

TL;DR

A dataset of 3,000 phacoemulsification cataract surgery videos from two surgical centers, performed by surgeons with a range of experience levels is presented, establishing a domain adaptation baseline for the phase recognition task by training a model on a subset of surgical centers and evaluating its performance on a held-out center.

Abstract

The development of computer-assisted surgery systems depends on large-scale, annotated datasets. Current resources for cataract surgery often lack the diversity and annotation depth needed to train generalizable deep-learning models. To address this gap, we present a dataset of 3,000 phacoemulsification cataract surgery videos from two surgical centers, performed by surgeons with a range of experience levels. This resource is enriched with four annotation layers: temporal surgical phases, instance segmentation of instruments and anatomical structures, instrument-tissue interaction tracking, and quantitative skill scores based on the established competency rubrics like the ICO-OSCAR. The technical quality of the dataset is supported by a series of benchmarking experiments for key surgical AI tasks, including workflow recognition, scene segmentation, and automated skill assessment. Furthermore, we establish a domain adaptation baseline for the phase recognition task by training a model on a subset of surgical centers and evaluating its performance on a held-out center. The dataset and annotations are available in Google Form (https://docs.google.com/forms/d/e/1FAIpQLSfmyMAPSTGrIy2sTnz0-TMw08ZagTimRulbAQcWdaPwDy187A/viewform?usp=dialog).

Figures (11)

• Figure 1: Visual overview of key surgical phases from both clinical centers, illustrating domain shift.
• Figure 2: Distribution of total time spent in each surgical phase across the 150 annotated videos.
• Figure 3: Normalized timelines illustrating procedural heterogeneity across 150 surgeries. Each row represents a single surgery, with phase transitions color-coded, normalized to a standard length from 0 (start) to 1 (end).
• Figure 4: Examples of surgical instruments from the two data sources, illustrating domain shift.
• Figure 5: Examples of common visual challenges for instance segmentation in the dataset.