Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Intraoperative Glioma Segmentation with YOLO + SAM for Improved Accuracy in Tumor Resection

Samir Kassam, Angelo Markham, Katie Vo, Yashas Revanakara, Michael Lam, Kevin Zhu

TL;DR

This work addresses real-time intraoperative glioma segmentation under ioMRI-like conditions, where brain shift and image noise hinder accurate delineation. It introduces a two-stage pipeline that uses YOLOv8 for rapid tumor detection followed by SAM ViT-b for precise segmentation, trained on BraTS 2021 data with Gaussian-noise augmentation to mimic ioMRI. The approach achieves a Dice score of $0.79$ on augmented data with fast inference ($15$–$25$ seconds), demonstrating robustness to noisier imaging and competitive performance against noiseless-data baselines. The results suggest potential clinical value for guiding tumor resection in real time, with future work aimed at incorporating true clinical ioMRI data and extending SAM to 3D contexts (e.g., TomoSAM) to retain volumetric information.

Abstract

Gliomas, a common type of malignant brain tumor, present significant surgical challenges due to their similarity to healthy tissue. Preoperative Magnetic Resonance Imaging (MRI) images are often ineffective during surgery due to factors such as brain shift, which alters the position of brain structures and tumors. This makes real-time intraoperative MRI (ioMRI) crucial, as it provides updated imaging that accounts for these shifts, ensuring more accurate tumor localization and safer resections. This paper presents a deep learning pipeline combining You Only Look Once Version 8 (YOLOv8) and Segment Anything Model Vision Transformer-base (SAM ViT-b) to enhance glioma detection and segmentation during ioMRI. Our model was trained using the Brain Tumor Segmentation 2021 (BraTS 2021) dataset, which includes standard magnetic resonance imaging (MRI) images, and noise-augmented MRI images that simulate ioMRI images. Noised MRI images are harder for a deep learning pipeline to segment, but they are more representative of surgical conditions. Achieving a Dice Similarity Coefficient (DICE) score of 0.79, our model performs comparably to state-of-the-art segmentation models tested on noiseless data. This performance demonstrates the model's potential to assist surgeons in maximizing tumor resection and improving surgical outcomes.

Intraoperative Glioma Segmentation with YOLO + SAM for Improved Accuracy in Tumor Resection

TL;DR

This work addresses real-time intraoperative glioma segmentation under ioMRI-like conditions, where brain shift and image noise hinder accurate delineation. It introduces a two-stage pipeline that uses YOLOv8 for rapid tumor detection followed by SAM ViT-b for precise segmentation, trained on BraTS 2021 data with Gaussian-noise augmentation to mimic ioMRI. The approach achieves a Dice score of on augmented data with fast inference (– seconds), demonstrating robustness to noisier imaging and competitive performance against noiseless-data baselines. The results suggest potential clinical value for guiding tumor resection in real time, with future work aimed at incorporating true clinical ioMRI data and extending SAM to 3D contexts (e.g., TomoSAM) to retain volumetric information.

Abstract

Gliomas, a common type of malignant brain tumor, present significant surgical challenges due to their similarity to healthy tissue. Preoperative Magnetic Resonance Imaging (MRI) images are often ineffective during surgery due to factors such as brain shift, which alters the position of brain structures and tumors. This makes real-time intraoperative MRI (ioMRI) crucial, as it provides updated imaging that accounts for these shifts, ensuring more accurate tumor localization and safer resections. This paper presents a deep learning pipeline combining You Only Look Once Version 8 (YOLOv8) and Segment Anything Model Vision Transformer-base (SAM ViT-b) to enhance glioma detection and segmentation during ioMRI. Our model was trained using the Brain Tumor Segmentation 2021 (BraTS 2021) dataset, which includes standard magnetic resonance imaging (MRI) images, and noise-augmented MRI images that simulate ioMRI images. Noised MRI images are harder for a deep learning pipeline to segment, but they are more representative of surgical conditions. Achieving a Dice Similarity Coefficient (DICE) score of 0.79, our model performs comparably to state-of-the-art segmentation models tested on noiseless data. This performance demonstrates the model's potential to assist surgeons in maximizing tumor resection and improving surgical outcomes.
Paper Structure (8 sections, 3 figures)

This paper contains 8 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Methodology
  3. Data Preprocessing
  4. Architecture
  5. Training
  6. Results
  7. Discussion
  8. Acknowledgements

Figures (3)

  • Figure 1: Left: regular MRI image. Right: augmented MRI image with SNR of 10
  • Figure 2: YOLO + SAM architecture; grayscale images are processed through an RGB assignment function, then passed through YOLO in which a bounding box is located around the tumor, the middle coordinate of the bounding box is passed into SAM, finally SAM produces a segmented brain image.
  • Figure 3: DICE performance comparison (left) and inference times of various models (right).