A Review on Organ Deformation Modeling Approaches for Reliable Surgical Navigation using Augmented Reality

TL;DR

This review systematically maps organ deformation modeling approaches for AR-guided surgery, classifying them into model-based, data-driven, and hybrid families and detailing subcategories such as biomechanical FE models, atlas-based methods, and physics-guided DL. It surveys applications across hepatobiliary, brain, breast, spine, vascular, and renal surgeries, highlighting domain-specific challenges, performance metrics (e.g., TRE, HD, RMS error), and representative accuracy ranges. A key takeaway is the trade-off: FE-based methods offer physical fidelity but require difficult-to-obtain boundary data and can be computationally heavy, while data-driven and hybrid approaches enable real-time inference but rely on data quality and regularization to ensure generalizability. The paper emphasizes the need for data-efficient learning, human-in-the-loop strategies to improve correspondences, rigorous validation, and bias mitigation to translate deformation modeling into reliable, safe AR-guided interventions.

Abstract

Augmented Reality (AR) holds the potential to revolutionize surgical procedures by allowing surgeons to visualize critical structures within the patient's body. This is achieved through superimposing preoperative organ models onto the actual anatomy. Challenges arise from dynamic deformations of organs during surgery, making preoperative models inadequate for faithfully representing intraoperative anatomy. To enable reliable navigation in augmented surgery, modeling of intraoperative deformation to obtain an accurate alignment of the preoperative organ model with the intraoperative anatomy is indispensable. Despite the existence of various methods proposed to model intraoperative organ deformation, there are still few literature reviews that systematically categorize and summarize these approaches. This review aims to fill this gap by providing a comprehensive and technical-oriented overview of modeling methods for intraoperative organ deformation in augmented reality in surgery. Through a systematic search and screening process, 112 closely relevant papers were included in this review. By presenting the current status of organ deformation modeling methods and their clinical applications, this review seeks to enhance the understanding of organ deformation modeling in AR-guided surgery, and discuss the potential topics for future advancements.

Figures (10)

• Figure 1: Surgical navigation using augmented reality with an illustrative example of hepatectomy zhang2021augmented: A: preoperative CT images containing the liver; B: 3D reconstructed organ models for liver; C: residual liver volume from simulated right hepatectomy; D: intraoperative navigation of the hepatic artery; E: intraoperative navigation of the portal vein; F: intraoperative navigation of the hepatic vein. Permissions: images licensed from the reference zhang2021augmented under CC BY 4.0.
• Figure 2: The workflow of the screening process using PRISMA page2021prisma protocol for the performed review.
• Figure 3: Illustration of several common intraoperative observations. A: reconstructed organ surface (right) from laparoscopic images (left); B: the position of anatomical points (red points) manually picked by surgeons on Ultrasound images; C: optical markers attach on the abdomen (left) and instruments (right). Permissions: some illustrative images reprinted from representative references maier2013opticalmaier2013opticalbernhardt2017status under respective copyright license permission from Elsevier.
• Figure 4: Overview of deformation modeling methods: (a) Model-based methods include: geometry-based alignment methods, biomechanical models, statistical models, and physics-based modeling. (b) Data-driven methods include: alras-based modeling, and machine learning approaches. (c) Hybrid methods include: network-driven biomechanical models, physics-guided deep learning, and multi-stage deformation modeling methods. Permissions: some illustrative images reprinted from representative references wang2023bonemendizabal2020simulationjud2017statisticaldagon2008frameworklabrunie2023automaticpawar2021physicsheiselman2024imagemarchesseau2017nonlinear under respective copyright license permission from Elsevier, IEEE, CC BY 4.0, or AIP Publishing.
• Figure 5: Annual distribution of articles focusing on organ deformation modeling methods.