OphNet: A Large-Scale Video Benchmark for Ophthalmic Surgical Workflow Understanding

TL;DR

OphNet introduces a large-scale, expert-annotated ophthalmic surgical video benchmark to enable comprehensive understanding of surgical workflows. It provides 2,278 videos (284.8 hours) across 66 surgeries with 102 phases and 150 operations, along with time-localized, hierarchical annotations and 523 localization-ready videos. The paper benchmarks four task families—presence, recognition, localization, and anticipation—using state-of-the-art models, establishing strong baselines and revealing task-specific performance trends. By addressing dataset scale, diversity, and annotation granularity, OphNet aims to advance AI-assisted ophthalmic surgery, documentation, and training. It also outlines practical extension directions like weak supervision, few-shot learning, and domain generalization to broaden applicability across clinical settings.

Abstract

Surgical scene perception via videos is critical for advancing robotic surgery, telesurgery, and AI-assisted surgery, particularly in ophthalmology. However, the scarcity of diverse and richly annotated video datasets has hindered the development of intelligent systems for surgical workflow analysis. Existing datasets face challenges such as small scale, lack of diversity in surgery and phase categories, and absence of time-localized annotations. These limitations impede action understanding and model generalization validation in complex and diverse real-world surgical scenarios. To address this gap, we introduce OphNet, a large-scale, expert-annotated video benchmark for ophthalmic surgical workflow understanding. OphNet features: 1) A diverse collection of 2,278 surgical videos spanning 66 types of cataract, glaucoma, and corneal surgeries, with detailed annotations for 102 unique surgical phases and 150 fine-grained operations. 2) Sequential and hierarchical annotations for each surgery, phase, and operation, enabling comprehensive understanding and improved interpretability. 3) Time-localized annotations, facilitating temporal localization and prediction tasks within surgical workflows. With approximately 285 hours of surgical videos, OphNet is about 20 times larger than the largest existing surgical workflow analysis benchmark. Code and dataset are available at: https://minghu0830.github.io/OphNet-benchmark/.

Figures (8)

• Figure 1: The figure shows two combined surgical videos, PHACO + IOL implantation and PHACO + Trabeculotomy + Transscleral Cyclophotocoagulation + IOL implantation. For each frame marked in color, we provide time-boundary annotations at surgical, phase and operation levels.
• Figure 2: OphNet's composition, comparison with other datasets for the same task, and some phase examples: (a) an overview of the composition ratios at the levels of surgery, phase, and operation; (b) comparison among existing open-source laparoscopic & endoscopic, and ophthalmic microscope workflow analysis video datasets and our OphNet. OphNet stands as the largest real-world video dataset for ophthalmic surgical workflow understanding, featuring the highest number of videos, longest duration, and diverse categories of surgeries and phases; (c) eight phase examples in OphNet.
• Figure 3: We present the data statistics of trimmed videos at the levels of phase and operation, including the number of trimmed videos, average duration, and total duration. The IDs and corresponding names can be found in the appendix.
• Figure 4: Attention map visualizations of ViFi-CLIP hanoonavificlip on four examples from OphNet's test set in the phase recognition task.
• Figure 5: Phase localization visualization of TriDet shi2023tridet. GT represents the ground truth visualization for phases, while Loc. visualizes the model's highest confidence phase category and the time-boundary results. Blank segments denote invalid segments or operation gaps.