Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TractoGPT: A GPT architecture for White Matter Segmentation

Anoushkrit Goel, Simroop Singh, Ankita Joshi, Ranjeet Ranjan Jha, Chirag Ahuja, Aditya Nigam, Arnav Bhavsar

TL;DR

TractoGPT addresses white matter tract segmentation from diffusion MRI by deploying a GPT-based autoregressive model trained on multiple tractography representations (Streamline, Cluster, Fusion). It introduces Fusion Data representation, tokenization via FPS-kNN patches, and a transformer-based Extractor-Generator that reconstructs patches and classifies streamlines, achieving state-of-the-art DICE, Overlap, and Overreach on TractoInferno and cross-dataset generalization to HCP (TractoGPT-hcp). Ablation shows Fusion and Cluster representations deliver strong performance, while cross-dataset results demonstrate robustness. The work offers a fully automatic, registration-free approach that preserves bundle shape, with efficient training and inference suitable for clinical and research workflows.

Abstract

White matter bundle segmentation is crucial for studying brain structural connectivity, neurosurgical planning, and neurological disorders. White Matter Segmentation remains challenging due to structural similarity in streamlines, subject variability, symmetry in 2 hemispheres, etc. To address these challenges, we propose TractoGPT, a GPT-based architecture trained on streamline, cluster, and fusion data representations separately. TractoGPT is a fully-automatic method that generalizes across datasets and retains shape information of the white matter bundles. Experiments also show that TractoGPT outperforms state-of-the-art methods on average DICE, Overlap and Overreach scores. We use TractoInferno and 105HCP datasets and validate generalization across dataset.

TractoGPT: A GPT architecture for White Matter Segmentation

TL;DR

TractoGPT addresses white matter tract segmentation from diffusion MRI by deploying a GPT-based autoregressive model trained on multiple tractography representations (Streamline, Cluster, Fusion). It introduces Fusion Data representation, tokenization via FPS-kNN patches, and a transformer-based Extractor-Generator that reconstructs patches and classifies streamlines, achieving state-of-the-art DICE, Overlap, and Overreach on TractoInferno and cross-dataset generalization to HCP (TractoGPT-hcp). Ablation shows Fusion and Cluster representations deliver strong performance, while cross-dataset results demonstrate robustness. The work offers a fully automatic, registration-free approach that preserves bundle shape, with efficient training and inference suitable for clinical and research workflows.

Abstract

White matter bundle segmentation is crucial for studying brain structural connectivity, neurosurgical planning, and neurological disorders. White Matter Segmentation remains challenging due to structural similarity in streamlines, subject variability, symmetry in 2 hemispheres, etc. To address these challenges, we propose TractoGPT, a GPT-based architecture trained on streamline, cluster, and fusion data representations separately. TractoGPT is a fully-automatic method that generalizes across datasets and retains shape information of the white matter bundles. Experiments also show that TractoGPT outperforms state-of-the-art methods on average DICE, Overlap and Overreach scores. We use TractoInferno and 105HCP datasets and validate generalization across dataset.
Paper Structure (12 sections, 3 figures, 3 tables)

This paper contains 12 sections, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. dMRI Datasets and Tractography
  4. Data Representations
  5. Tokenization
  6. TractoGPT Model
  7. Model Training and Testing
  8. Experiments and Results
  9. Conclusion
  10. Compliance with ethical standards
  11. Acknowledgments
  12. References

Figures (3)

  • Figure 1: TractoGPT Architecture:(Stage I) Raw Streamline undergoes preprocessing (i, ii, iii, iv) to give us 3 different data representations (Section \ref{['sec:preprocess']}). (Stage II) From 3 different point cloud arrays, any one can be chosen to train TractoGPT, Extracted Point Cloud undergoes FPS (Farthest Point Sampling) to give total $P$ center points (Absolute Positions), used to sample a total of $K$ nearest neighbors using kNN (v), creating point cloud patches. (Stage III) Point patches get sequence using Morton order (Relative Positions) (vi), and a PointNet-style encoder gives embedding for each patch as tokens.
  • Figure 2: Voxel DICE scores for class-wise comparison across FINTA-mRecoBundlesX, TractoGPT-hcp, TractoGPT methods on TractoInferno test dataset. Class-wise Ablation study of TractoGPT are across [streamline, cluster, fusion] data representations. Here TractoGPT-hcp results are shown for dataset generalization, which is trained on HCP and tested on TractoInferno
  • Figure 3: Visualisation of major bundles, tested on sub-1006