Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Patient-Specific Real-Time Segmentation in Trackerless Brain Ultrasound

Reuben Dorent, Erickson Torio, Nazim Haouchine, Colin Galvin, Sarah Frisken, Alexandra Golby, Tina Kapur, William Wells

TL;DR

This work tackles the challenge of interpreting trackerless brain intraoperative ultrasound (iUS) by proposing a patient-specific, real-time segmentation framework trained with synthetic iUS generated from pre-operative MR data. It introduces a three-step pipeline: simulate probe sweeps on the brain surface, synthesize iUS from MR data with a Multi-Modal Hierarchical Variational Auto-Encoder (MHVAE), and train a subject-specific segmentation network $f_{\theta^s}$ using a large, varied synthetic dataset. Through experiments on seven ReMIND cases, the approach outperforms non-patient-specific baselines and matches or surpasses tracking-based methods, even outperforming an experienced iUS expert in some cases, demonstrating the value of patient-specific pre-operative planning for iUS interpretation. The results suggest that trackerless, MR-guided iUS segmentation can reduce reliance on expensive navigation systems while enabling accurate, real-time surgical targeting.

Abstract

Intraoperative ultrasound (iUS) imaging has the potential to improve surgical outcomes in brain surgery. However, its interpretation is challenging, even for expert neurosurgeons. In this work, we designed the first patient-specific framework that performs brain tumor segmentation in trackerless iUS. To disambiguate ultrasound imaging and adapt to the neurosurgeon's surgical objective, a patient-specific real-time network is trained using synthetic ultrasound data generated by simulating virtual iUS sweep acquisitions in pre-operative MR data. Extensive experiments performed in real ultrasound data demonstrate the effectiveness of the proposed approach, allowing for adapting to the surgeon's definition of surgical targets and outperforming non-patient-specific models, neurosurgeon experts, and high-end tracking systems. Our code is available at: \url{https://github.com/ReubenDo/MHVAE-Seg}.

Patient-Specific Real-Time Segmentation in Trackerless Brain Ultrasound

TL;DR

This work tackles the challenge of interpreting trackerless brain intraoperative ultrasound (iUS) by proposing a patient-specific, real-time segmentation framework trained with synthetic iUS generated from pre-operative MR data. It introduces a three-step pipeline: simulate probe sweeps on the brain surface, synthesize iUS from MR data with a Multi-Modal Hierarchical Variational Auto-Encoder (MHVAE), and train a subject-specific segmentation network using a large, varied synthetic dataset. Through experiments on seven ReMIND cases, the approach outperforms non-patient-specific baselines and matches or surpasses tracking-based methods, even outperforming an experienced iUS expert in some cases, demonstrating the value of patient-specific pre-operative planning for iUS interpretation. The results suggest that trackerless, MR-guided iUS segmentation can reduce reliance on expensive navigation systems while enabling accurate, real-time surgical targeting.

Abstract

Intraoperative ultrasound (iUS) imaging has the potential to improve surgical outcomes in brain surgery. However, its interpretation is challenging, even for expert neurosurgeons. In this work, we designed the first patient-specific framework that performs brain tumor segmentation in trackerless iUS. To disambiguate ultrasound imaging and adapt to the neurosurgeon's surgical objective, a patient-specific real-time network is trained using synthetic ultrasound data generated by simulating virtual iUS sweep acquisitions in pre-operative MR data. Extensive experiments performed in real ultrasound data demonstrate the effectiveness of the proposed approach, allowing for adapting to the surgeon's definition of surgical targets and outperforming non-patient-specific models, neurosurgeon experts, and high-end tracking systems. Our code is available at: \url{https://github.com/ReubenDo/MHVAE-Seg}.
Paper Structure (18 sections, 2 equations, 5 figures, 2 tables)

This paper contains 18 sections, 2 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Patient-specific segmentation in brain ultrasound
  3. Problem overview.
  4. Step 1: Simulation of ultrasound acquisitions
  5. Step 2: Synthesizing ultrasound data from MR data
  6. Step 3: Training a patient-specific segmentation network
  7. Training set.
  8. Network training.
  9. Experiments
  10. Dataset.
  11. Pre-operative segmentations.
  12. Evaluation.
  13. Determining the number of virtual sweeps.
  14. Baseline approaches.
  15. Results.
  16. ...and 3 more sections

Figures (5)

  • Figure 1: An illustrative example showing a) the variability between neurosurgeon and BraTS annotation protocols; b) the challenge of identifying tumor boundaries in untracked 2D iUS; c) misalignment overlay issues of tracked 2D iUS with navigation system (red arrows); d) predictions of our patient-specific models in untracked 2D iUS.
  • Figure 2: Overview of our approach to create synthetic iUS images from pre-operative MRI. First, a generic sweep is virtually positioned on MR brain surface to simulate iUS acquisition. Then, MHVAE dorent2023unified synthesizes iUS from generated 2D pre-op MRI slices.
  • Figure 3: Examples of automated surgical target segmentations using non-patient-specific models (RESECT Unet and BraTS Unet), high-end tracking system and Ours. MR scan used for manual surgical target annotation is shown.
  • Figure 4: Impact of the temperature $\tau$ on the local variability (speckles).
  • Figure 5: Our 2D Unet segmentation network.