Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Decoder-Free Supervoxel GNN for Accurate Brain-Tumor Localization in Multi-Modal MRI

Andrea Protani, Marc Molina Van Den Bosch, Lorenzo Giusti, Heloisa Barbosa Da Silva, Paolo Cacace, Albert Sund Aillet, Miguel Angel Gonzalez Ballester, Friedhelm Hummel, Luigi Serio

TL;DR

This work tackles the inefficiency and limited interpretability of decoder-heavy 3D medical image models by proposing SVGFormer, a decoder-free pipeline that casts multi-modal MRI as a hierarchical supervoxel graph. A patch-level Transformer embeds local voxel information within each supervoxel, which is then refined by a Graph Attention Network to capture inter-regional context, enabling dual-scale explainability from patch features to region interactions. The authors demonstrate that this encoder-centric approach supports both node-level tumor-classification and tumor-proportion regression on BraTS, achieving strong metrics (e.g., F1 ~0.87, MAE ~0.028) and robust generalization to downstream segmentation without extra finetuning. The framework offers a flexible, multi-task backbone with inherent interpretability, promising more efficient training and clearer decision paths for clinical deployment and future extension to other tasks.

Abstract

Modern vision backbones for 3D medical imaging typically process dense voxel grids through parameter-heavy encoder-decoder structures, a design that allocates a significant portion of its parameters to spatial reconstruction rather than feature learning. Our approach introduces SVGFormer, a decoder-free pipeline built upon a content-aware grouping stage that partitions the volume into a semantic graph of supervoxels. Its hierarchical encoder learns rich node representations by combining a patch-level Transformer with a supervoxel-level Graph Attention Network, jointly modeling fine-grained intra-region features and broader inter-regional dependencies. This design concentrates all learnable capacity on feature encoding and provides inherent, dual-scale explainability from the patch to the region level. To validate the framework's flexibility, we trained two specialized models on the BraTS dataset: one for node-level classification and one for tumor proportion regression. Both models achieved strong performance, with the classification model achieving a F1-score of 0.875 and the regression model a MAE of 0.028, confirming the encoder's ability to learn discriminative and localized features. Our results establish that a graph-based, encoder-only paradigm offers an accurate and inherently interpretable alternative for 3D medical image representation.

Decoder-Free Supervoxel GNN for Accurate Brain-Tumor Localization in Multi-Modal MRI

TL;DR

This work tackles the inefficiency and limited interpretability of decoder-heavy 3D medical image models by proposing SVGFormer, a decoder-free pipeline that casts multi-modal MRI as a hierarchical supervoxel graph. A patch-level Transformer embeds local voxel information within each supervoxel, which is then refined by a Graph Attention Network to capture inter-regional context, enabling dual-scale explainability from patch features to region interactions. The authors demonstrate that this encoder-centric approach supports both node-level tumor-classification and tumor-proportion regression on BraTS, achieving strong metrics (e.g., F1 ~0.87, MAE ~0.028) and robust generalization to downstream segmentation without extra finetuning. The framework offers a flexible, multi-task backbone with inherent interpretability, promising more efficient training and clearer decision paths for clinical deployment and future extension to other tasks.

Abstract

Modern vision backbones for 3D medical imaging typically process dense voxel grids through parameter-heavy encoder-decoder structures, a design that allocates a significant portion of its parameters to spatial reconstruction rather than feature learning. Our approach introduces SVGFormer, a decoder-free pipeline built upon a content-aware grouping stage that partitions the volume into a semantic graph of supervoxels. Its hierarchical encoder learns rich node representations by combining a patch-level Transformer with a supervoxel-level Graph Attention Network, jointly modeling fine-grained intra-region features and broader inter-regional dependencies. This design concentrates all learnable capacity on feature encoding and provides inherent, dual-scale explainability from the patch to the region level. To validate the framework's flexibility, we trained two specialized models on the BraTS dataset: one for node-level classification and one for tumor proportion regression. Both models achieved strong performance, with the classification model achieving a F1-score of 0.875 and the regression model a MAE of 0.028, confirming the encoder's ability to learn discriminative and localized features. Our results establish that a graph-based, encoder-only paradigm offers an accurate and inherently interpretable alternative for 3D medical image representation.
Paper Structure (17 sections, 3 figures, 1 table)

This paper contains 17 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background and Related Works
  3. Deep Learning on Volumetric Brain MRI.
  4. Transformer-based Encoders for Medical Images.
  5. Graph Neural Networks for Irregular Domains.
  6. Supervoxel-Based Representations.
  7. Supervoxel Graph Transformer
  8. Pre-processing Pipeline.
  9. Patch-Graph Neural Network Architecture.
  10. Experiments
  11. Dataset.
  12. Implementation Details.
  13. Evaluation Metrics.
  14. Experimental Results.
  15. Model Performance.
  16. ...and 2 more sections

Figures (3)

  • Figure 1: Pipeline for supervoxel (SV) graph 3D-image encoding: (a) Initial MRI modalities; (b) Apply SLIC to T1-WI (SV grid shared across co-registered modalities); (c) Reconstructed graph of SVs as nodes connected to their 8 nearest neighbors; (d) Patchify method, relying on SV sampling of flattened voxel patches. While the graph is dense and packed, for visualization purposes a sparse graph is represented.
  • Figure 2: Architecture of the SVGFormer; Hierarchical Encoder and Regressor. (i) Node Embedder: modality-aware patch tokens pass through a 5-layer Transformer to form node features. (ii) Graph Encoder: a 5-layer GATv2, augmented with Laplacian positional encodings, propagates context across supervoxels. (iii) Regressor: an attention-weighted ensemble of Multi-Layer Perceptron (MLP) heads predicts the tumor fraction for each node.
  • Figure 3: Qualitative analysis of model predictions on four different patients. The top row shows results from binarized Classification models, while the bottom row shows predictions from the Regression models. From left to right, the columns correspond to models trained with $\mathit{1000}$, $\mathit{2000}$, $\mathit{3000}$, and $\mathit{4000}$supervoxels. Each of these eight cells is itself a $2\times 2$ quadrant, displaying a sagittal view of the model's prediction for each of the four patients (identified as A-D at the corner).