Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TrackletGPT: A Language-like GPT Framework for White Matter Tract Segmentation

Anoushkrit Goel, Simroop Singh, Ankita Joshi, Ranjeet Ranjan Jha, Chirag Ahuja, Aditya Nigam, Arnav Bhavsar

TL;DR

TrackletGPT reframes white matter tract segmentation as a language-modeling problem by encoding streamline subsequences as Tracklets, sentences, and paragraphs. It combines a PointNet-style encoder with a GPT-like autoregressive transformer, using dual-masking pretraining and a lightweight architecture to achieve registration-free, cross-dataset segmentation across TractoInferno and HCP. The key contributions include the Tracklets representation, the multi-level (Tracklets–sentences–Paragraphs) framing, and state-of-the-art inter-dataset performance with ablations supporting the benefits of paragraph-level context and sinusoidal positional encoding. This approach offers a scalable foundation for foundation models in tractography, enabling robust segmentation without atlas reliance and improving generalization across subjects and datasets.

Abstract

White Matter Tract Segmentation is imperative for studying brain structural connectivity, neurological disorders and neurosurgery. This task remains complex, as tracts differ among themselves, across subjects and conditions, yet have similar 3D structure across hemispheres and subjects. To address these challenges, we propose TrackletGPT, a language-like GPT framework which reintroduces sequential information in tokens using tracklets. TrackletGPT generalises seamlessly across datasets, is fully automatic, and encodes granular sub-streamline segments, Tracklets, scaling and refining GPT models in Tractography Segmentation. Based on our experiments, TrackletGPT outperforms state-of-the-art methods on average DICE, Overlap and Overreach scores on TractoInferno and HCP datasets, even on inter-dataset experiments.

TrackletGPT: A Language-like GPT Framework for White Matter Tract Segmentation

TL;DR

TrackletGPT reframes white matter tract segmentation as a language-modeling problem by encoding streamline subsequences as Tracklets, sentences, and paragraphs. It combines a PointNet-style encoder with a GPT-like autoregressive transformer, using dual-masking pretraining and a lightweight architecture to achieve registration-free, cross-dataset segmentation across TractoInferno and HCP. The key contributions include the Tracklets representation, the multi-level (Tracklets–sentences–Paragraphs) framing, and state-of-the-art inter-dataset performance with ablations supporting the benefits of paragraph-level context and sinusoidal positional encoding. This approach offers a scalable foundation for foundation models in tractography, enabling robust segmentation without atlas reliance and improving generalization across subjects and datasets.

Abstract

White Matter Tract Segmentation is imperative for studying brain structural connectivity, neurological disorders and neurosurgery. This task remains complex, as tracts differ among themselves, across subjects and conditions, yet have similar 3D structure across hemispheres and subjects. To address these challenges, we propose TrackletGPT, a language-like GPT framework which reintroduces sequential information in tokens using tracklets. TrackletGPT generalises seamlessly across datasets, is fully automatic, and encodes granular sub-streamline segments, Tracklets, scaling and refining GPT models in Tractography Segmentation. Based on our experiments, TrackletGPT outperforms state-of-the-art methods on average DICE, Overlap and Overreach scores on TractoInferno and HCP datasets, even on inter-dataset experiments.
Paper Structure (12 sections, 3 figures, 4 tables)

This paper contains 12 sections, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. dMRI Datasets and Tractography
  4. Data Representation
  5. Model Input
  6. TrackletGPT Model
  7. Model Training and Testing
  8. Experiments and Results
  9. Conclusion
  10. Compliance with ethical standards
  11. Acknowledgments
  12. References

Figures (3)

  • Figure 1: TrackletGPT Framework: (I) Tracklets are formed by connecting inflection points obtained from downsampling interpolation, (II) Raw streamline can either (i) form sentences or (ii) form paragraphs along with neighbouring streamlines. Sequenced sentences(iii) are randomly ordered aggregate to $n$ tracklets where each bicubic-interpolated (12,3) tracklet is (iv) encoded by Pointnet-style encoder to give $(n_{tokens}, t_{dim})$. These tokens along with (v) positional embedding are sent to the GPT framework to reconstruct tracklet tokens.
  • Figure 2: Voxel DICE scores for class-wise comparison across FINTA-mRecoBundlesX, TrackletGPT, TractoGPT methods on TractoInferno and HCP, showcasing results on same (self), different (inter-dataset) and combined train test datasets.
  • Figure 3: Visualisation of Inter-dataset test results for sub-1006 (Tractoinferno) & 599469 (HCP)