SurgBox: Agent-Driven Operating Room Sandbox with Surgery Copilot

TL;DR

Neurosurgical interventions impose high cognitive demands on teams, and patient safety limits real-world deliberate practice. The authors introduce SurgBox, an agent-driven sandbox that uses LLM-based role agents with tailored Retrieval-Augmented Generation to simulate operating room dynamics and a Surgery Copilot to coordinate information streams in real time. A Long-Short Memory mechanism enables the Copilot to blend immediate procedural guidance with broad surgical knowledge, while real neurosurgery records from $128$ patients validate the approach, achieving $88.00\%$ route accuracy and $88.02\%$ plan accuracy and outperforming baselines. The work offers a practical framework for training and operational support that could improve cognitive performance and potentially patient outcomes in complex surgeries.

Abstract

Surgical interventions, particularly in neurology, represent complex and high-stakes scenarios that impose substantial cognitive burdens on surgical teams. Although deliberate education and practice can enhance cognitive capabilities, surgical training opportunities remain limited due to patient safety concerns. To address these cognitive challenges in surgical training and operation, we propose SurgBox, an agent-driven sandbox framework to systematically enhance the cognitive capabilities of surgeons in immersive surgical simulations. Specifically, our SurgBox leverages large language models (LLMs) with tailored Retrieval-Augmented Generation (RAG) to authentically replicate various surgical roles, enabling realistic training environments for deliberate practice. In particular, we devise Surgery Copilot, an AI-driven assistant to actively coordinate the surgical information stream and support clinical decision-making, thereby diminishing the cognitive workload of surgical teams during surgery. By incorporating a novel Long-Short Memory mechanism, our Surgery Copilot can effectively balance immediate procedural assistance with comprehensive surgical knowledge. Extensive experiments using real neurosurgical procedure records validate our SurgBox framework in both enhancing surgical cognitive capabilities and supporting clinical decision-making. By providing an integrated solution for training and operational support to address cognitive challenges, our SurgBox framework advances surgical education and practice, potentially transforming surgical outcomes and healthcare quality. The code is available at https://github.com/franciszchen/SurgBox.

Figures (10)

• Figure 1: The overview of the SurgBox framework. In the simulated operating room sandbox, each surgical role is automatically driven by an LLM-based agent. In particular, we devise the Surgical Copilot that is responsible for planning and guiding the entire surgical process.
• Figure 2: Examples of surgical roles in SurgBox include the chief surgeon, surgeon assistant, scrub nurse, and Surgery Copilot.
• Figure 3: The surgical workflow in the SurgBox framework. The SurgBox framework simulates the patient's entire surgical closed loop, including patient transfer, anesthesia, preparation, surgical operation, and postoperative care.
• Figure 4: The interaction of Surgery Copilot and surgical roles in the surgical workflow.
• Figure 5: The distribution of neurosurgical diseases and neurosurgery routes in the real neurosurgical procedure records.