Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SurGo-R1: Benchmarking and Modeling Contextual Reasoning for Operative Zone in Surgical Video

Guanyi Qin, Xiaozhen Wang, Zhu Zhuo, Chang Han Low, Yuancan Xiao, Yibing Fu, Haofeng Liu, Kai Wang, Chunjiang Li, Yueming Jin

TL;DR

ResGo, a benchmark of laparoscopic frames annotated with Go Zone bounding boxes and clinician-authored rationales covering phase, exposure quality reasoning, next action and risk reminder, and SurGo-R1, a model optimized via RLHF with a multi-turn phase-then-go architecture where the model first identifies the surgical phase, then generates reasoning and Go Zone coordinates conditioned on that context.

Abstract

Minimally invasive surgery has dramatically improved patient operative outcomes, yet identifying safe operative zones remains challenging in critical phases, requiring surgeons to integrate visual cues, procedural phase, and anatomical context under high cognitive load. Existing AI systems offer binary safety verification or static detection, ignoring the phase-dependent nature of intraoperative reasoning. We introduce ResGo, a benchmark of laparoscopic frames annotated with Go Zone bounding boxes and clinician-authored rationales covering phase, exposure quality reasoning, next action and risk reminder. We introduce evaluation metrics that treat correct grounding under incorrect phase as failures, revealing that most vision-language models cannot handle such tasks and perform poorly. We then present SurGo-R1, a model optimized via RLHF with a multi-turn phase-then-go architecture where the model first identifies the surgical phase, then generates reasoning and Go Zone coordinates conditioned on that context. On unseen procedures, SurGo-R1 achieves 76.6% phase accuracy, 32.7 mIoU, and 54.8% hardcore accuracy, a 6.6$\times$ improvement over the mainstream generalist VLMs. Code, model and benchmark will be available at https://github.com/jinlab-imvr/SurGo-R1

SurGo-R1: Benchmarking and Modeling Contextual Reasoning for Operative Zone in Surgical Video

TL;DR

ResGo, a benchmark of laparoscopic frames annotated with Go Zone bounding boxes and clinician-authored rationales covering phase, exposure quality reasoning, next action and risk reminder, and SurGo-R1, a model optimized via RLHF with a multi-turn phase-then-go architecture where the model first identifies the surgical phase, then generates reasoning and Go Zone coordinates conditioned on that context.

Abstract

Minimally invasive surgery has dramatically improved patient operative outcomes, yet identifying safe operative zones remains challenging in critical phases, requiring surgeons to integrate visual cues, procedural phase, and anatomical context under high cognitive load. Existing AI systems offer binary safety verification or static detection, ignoring the phase-dependent nature of intraoperative reasoning. We introduce ResGo, a benchmark of laparoscopic frames annotated with Go Zone bounding boxes and clinician-authored rationales covering phase, exposure quality reasoning, next action and risk reminder. We introduce evaluation metrics that treat correct grounding under incorrect phase as failures, revealing that most vision-language models cannot handle such tasks and perform poorly. We then present SurGo-R1, a model optimized via RLHF with a multi-turn phase-then-go architecture where the model first identifies the surgical phase, then generates reasoning and Go Zone coordinates conditioned on that context. On unseen procedures, SurGo-R1 achieves 76.6% phase accuracy, 32.7 mIoU, and 54.8% hardcore accuracy, a 6.6 improvement over the mainstream generalist VLMs. Code, model and benchmark will be available at https://github.com/jinlab-imvr/SurGo-R1
Paper Structure (21 sections, 1 equation, 5 figures, 6 tables)

This paper contains 21 sections, 1 equation, 5 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. ResGo Benchmark
  4. Source Acquisition and Demographics
  5. Annotation Framework and Pipeline
  6. Analysis and Splits of Annotated Samples
  7. Train and Test Splits
  8. Benchmark Problem Formulation
  9. Evaluation Protocols
  10. SurGo-R1 Baseline Methodology
  11. Overview of SurGo-R1
  12. Prerequisite Turn: Contextual Priming
  13. Reasoning Turn: Surgical Analysis
  14. Phase-Definition Mapping Tool
  15. Reward Modeling
  16. ...and 6 more sections

Figures (5)

  • Figure 1: Demo of the proposed ResGo dataset, a novel multimodal benchmark that covers phase recognition, Go Zone grounding, and safety reasoning. Built on ResGo, a GRPO-optimized VLM with context-aware multi-turn reasoning, named SurGo-R1, is further introduced, enabling safety-aware reasoning and Go Zone grounding across scenarios.
  • Figure 2: The left panels detail patient demographics and clinical profiles, including distributions for gender, age, ASA classification, BMI, and relevant medical history. The right panels present the processed surgical phase statistics, illustrating duration distributions, occurrence frequencies, and the phase transition matrix for the four defined phases.
  • Figure 3: Illustration of the annotation and review pipeline
  • Figure 4: Overview of SurGo-R1 and phase-then-go pipeline.
  • Figure 5: Qualitative results of reasoning and grounding. Models trained without $\mathbf{R}_{\text{reason}}$ are denoted as w/o Rea or w/o Reason. Yellow and green bounding boxes represent model predictions and ground truths, respectively.