TractRLFusion: A GPT-Based Multi-Critic Policy Fusion Framework for Fiber Tractography
Ankita Joshi, Ashutosh Sharma, Anoushkrit Goel, Ranjeet Ranjan Jha, Chirag Ahuja, Arnav Bhavsar, Aditya Nigam
TL;DR
TractRLFusion addresses the FP-FN trade-off in white matter tractography by fusing multiple RL policies through a GPT-based FusionNet. It introduces Episodic Data Selection to curate anatomically informed training data and Multi-Critic Policy Fine-Tuning to refine the fusion with a multi-critic feedback loop. Across TractoInferno, HCP, and ISMRM datasets, FusionNet outperforms individual RL policies and RL ensembles, achieving higher Dice and more favorable OL/OR trade-offs, and even surpassing TractSeg in some settings. The approach demonstrates strong cross-dataset generalization and offers a modular framework for integrating additional tractography policies.
Abstract
Tractography plays a pivotal role in the non-invasive reconstruction of white matter fiber pathways, providing vital information on brain connectivity and supporting precise neurosurgical planning. Although traditional methods relied mainly on classical deterministic and probabilistic approaches, recent progress has benefited from supervised deep learning (DL) and deep reinforcement learning (DRL) to improve tract reconstruction. A persistent challenge in tractography is accurately reconstructing white matter tracts while minimizing spurious connections. To address this, we propose TractRLFusion, a novel GPT-based policy fusion framework that integrates multiple RL policies through a data-driven fusion strategy. Our method employs a two-stage training data selection process for effective policy fusion, followed by a multi-critic fine-tuning phase to enhance robustness and generalization. Experiments on HCP, ISMRM, and TractoInferno datasets demonstrate that TractRLFusion outperforms individual RL policies as well as state-of-the-art classical and DRL methods in accuracy and anatomical reliability.