Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Toward a Surgeon-in-the-Loop Ophthalmic Robotic Apprentice using Reinforcement and Imitation Learning

Amr Gomaa, Bilal Mahdy, Niko Kleer, Antonio Krüger

TL;DR

This work tackles how autonomous ophthalmic robotic systems can adapt to individual surgeon preferences during cataract incision tasks by combining reinforcement learning with imitation learning in a curriculum-learning framework guided by visual inputs. It introduces a Unity-based, high-fidelity ocular simulator and a surgeon-in-the-loop training strategy that blends PPO with GAIL, allowing the agent to learn from demonstrations while maintaining autonomous optimization. The study demonstrates a sector-based adaptation approach, revealing a trade-off between achieving surgeon-specific techniques (AdSSR) and general task completion (SCR), and provides an open-source framework to support future development. The results indicate potential for online real-time learning and transfer to physical robotic systems, paving the way for personalized, microsurgical robotics across ophthalmology and beyond.

Abstract

Robot-assisted surgical systems have demonstrated significant potential in enhancing surgical precision and minimizing human errors. However, existing systems cannot accommodate individual surgeons' unique preferences and requirements. Additionally, they primarily focus on general surgeries (e.g., laparoscopy) and are unsuitable for highly precise microsurgeries, such as ophthalmic procedures. Thus, we propose an image-guided approach for surgeon-centered autonomous agents that can adapt to the individual surgeon's skill level and preferred surgical techniques during ophthalmic cataract surgery. Our approach trains reinforcement and imitation learning agents simultaneously using curriculum learning approaches guided by image data to perform all tasks of the incision phase of cataract surgery. By integrating the surgeon's actions and preferences into the training process, our approach enables the robot to implicitly learn and adapt to the individual surgeon's unique techniques through surgeon-in-the-loop demonstrations. This results in a more intuitive and personalized surgical experience for the surgeon while ensuring consistent performance for the autonomous robotic apprentice. We define and evaluate the effectiveness of our approach in a simulated environment using our proposed metrics and highlight the trade-off between a generic agent and a surgeon-centered adapted agent. Finally, our approach has the potential to extend to other ophthalmic and microsurgical procedures, opening the door to a new generation of surgeon-in-the-loop autonomous surgical robots. We provide an open-source simulation framework for future development and reproducibility at https://github.com/amrgomaaelhady/CataractAdaptSurgRobot.

Toward a Surgeon-in-the-Loop Ophthalmic Robotic Apprentice using Reinforcement and Imitation Learning

TL;DR

This work tackles how autonomous ophthalmic robotic systems can adapt to individual surgeon preferences during cataract incision tasks by combining reinforcement learning with imitation learning in a curriculum-learning framework guided by visual inputs. It introduces a Unity-based, high-fidelity ocular simulator and a surgeon-in-the-loop training strategy that blends PPO with GAIL, allowing the agent to learn from demonstrations while maintaining autonomous optimization. The study demonstrates a sector-based adaptation approach, revealing a trade-off between achieving surgeon-specific techniques (AdSSR) and general task completion (SCR), and provides an open-source framework to support future development. The results indicate potential for online real-time learning and transfer to physical robotic systems, paving the way for personalized, microsurgical robotics across ophthalmology and beyond.

Abstract

Robot-assisted surgical systems have demonstrated significant potential in enhancing surgical precision and minimizing human errors. However, existing systems cannot accommodate individual surgeons' unique preferences and requirements. Additionally, they primarily focus on general surgeries (e.g., laparoscopy) and are unsuitable for highly precise microsurgeries, such as ophthalmic procedures. Thus, we propose an image-guided approach for surgeon-centered autonomous agents that can adapt to the individual surgeon's skill level and preferred surgical techniques during ophthalmic cataract surgery. Our approach trains reinforcement and imitation learning agents simultaneously using curriculum learning approaches guided by image data to perform all tasks of the incision phase of cataract surgery. By integrating the surgeon's actions and preferences into the training process, our approach enables the robot to implicitly learn and adapt to the individual surgeon's unique techniques through surgeon-in-the-loop demonstrations. This results in a more intuitive and personalized surgical experience for the surgeon while ensuring consistent performance for the autonomous robotic apprentice. We define and evaluate the effectiveness of our approach in a simulated environment using our proposed metrics and highlight the trade-off between a generic agent and a surgeon-centered adapted agent. Finally, our approach has the potential to extend to other ophthalmic and microsurgical procedures, opening the door to a new generation of surgeon-in-the-loop autonomous surgical robots. We provide an open-source simulation framework for future development and reproducibility at https://github.com/amrgomaaelhady/CataractAdaptSurgRobot.
Paper Structure (18 sections, 8 figures, 2 tables, 1 algorithm)

This paper contains 18 sections, 8 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Surgical Microscopy and Robotics
  4. Reinforcement Learning (RL)
  5. Imitation Learning
  6. Ophthalmic Cataract Surgery
  7. Methodology
  8. Simulation Environment
  9. Curriculum Learning Agent
  10. Low Poly -- Simple Learning Task
  11. High Poly -- Complex Learning Task
  12. Learning Environment
  13. Results and Discussion
  14. Performance Metrics
  15. Simulated Scenarios
  16. ...and 3 more sections

Figures (8)

  • Figure 1: Our cataract surgery simulation model. Top: Cornea illustrative sectorization for personalized approach modeling. Bottom: The surgical tool performing an incision in the cornea (highlighted in red).
  • Figure 2: A 2.75mm Keratome Ophthalmic Knife mapped and digitalized in 3D by coca2013models.
  • Figure 3: Camera setup for comprehensive surgical scene perception. The top-view camera (Camera 1) provides an overhead vantage point, capturing the surgical field and the overall context. The upper-side view camera (Camera 2) offers a lateral perspective, enabling observation of side-specific details and depth perception. Lastly, the camera with a view in the upper corner (Camera 3) provides an angled viewpoint, enhancing spatial awareness and facilitating precise tool manipulation.
  • Figure 4: Left: Low Poly eye model, Right: High Poly eye model
  • Figure 5: Cumulative reward convergence using RL pretraining on a low-poly eye model versus using a GAIL agent directly on a high-poly eye model.
  • ...and 3 more figures