MultiCo3D: Multi-Label Voxel Contrast for One-Shot Incremental Segmentation of 3D Neuroimages

TL;DR

The paper tackles the challenge of one-shot incremental segmentation of white matter tracts in 3D diffusion MRI by introducing MultiCo3D, a multi-label voxel contrast framework. It jointly learns base and novel tracts through a four-module architecture that includes base tract uncertainty distillation, a novel tract segmentation module, a multi-label voxel contrast loss, and a dynamic loss weighting scheme, all initialized via LwF. The key contributions are the introduction of a multi-label voxel contrast mechanism that leverages inter-class label similarity to avoid conflicting pulls/pushes in overlapping tracts, a dedicated uncertainty-based distillation to preserve base knowledge, and a dynamic weighting strategy to balance losses during training. Empirical results on the HCP and Preto datasets across five incremental setups show substantial improvements over state-of-the-art methods, with ablation studies validating the importance of each component and qualitative visualizations confirming cleaner, more accurate tract segmentation. The approach has practical impact for studies with limited annotations and could enable more reliable analysis of WM connectivity in health and disease.

Abstract

3D neuroimages provide a comprehensive view of brain structure and function, aiding in precise localization and functional connectivity analysis. Segmentation of white matter (WM) tracts using 3D neuroimages is vital for understanding the brain's structural connectivity in both healthy and diseased states. One-shot Class Incremental Semantic Segmentation (OCIS) refers to effectively segmenting new (novel) classes using only a single sample while retaining knowledge of old (base) classes without forgetting. Voxel-contrastive OCIS methods adjust the feature space to alleviate the feature overlap problem between the base and novel classes. However, since WM tract segmentation is a multi-label segmentation task, existing single-label voxel contrastive-based methods may cause inherent contradictions. To address this, we propose a new multi-label voxel contrast framework called MultiCo3D for one-shot class incremental tract segmentation. Our method utilizes uncertainty distillation to preserve base tract segmentation knowledge while adjusting the feature space with multi-label voxel contrast to alleviate feature overlap when learning novel tracts and dynamically weighting multi losses to balance overall loss. We compare our method against several state-of-the-art (SOTA) approaches. The experimental results show that our method significantly enhances one-shot class incremental tract segmentation accuracy across five different experimental setups on HCP and Preto datasets.

Figures (5)

• Figure 1: The Conflict of Single-Label Voxel Contrast and Our Proposed Multi-Label Voxel Contrast method. (a) Multi-Label samples have semantic overlap. (b) The conflict of the current single-label voxel contrast method, e.g., CoinSeg, may mistakenly pull and push the same sample simultaneously due to the binary nature of positive and negative sample distinctions. (c) Rather than relying on binary positive-negative sample distinctions, our multi-label voxel contrast method adjusts feature space based on their label similarity. This allows us to pull semantically overlapping samples closer and avoid overly pushing or pulling them apart. The balance orbit represents the stable region of semantically overlapping voxels, calculated based on the label similarity between the anchor and the voxel itself. Multi-Label voxels on the orbit remain stable (neither pushed away nor pulled closer).
• Figure 2: The overview of our proposed method. (a) The known (base) and unknown (novel) tracts. (b) The 3D peaks are extracted from dMRI data after preprocessing. (c) Our proposed multi-label voxel-contrast framework.
• Figure 3: Visualization of segmentation results of three novel tracts: left middle longitudinal fascicle (MLF), left corticospinal tract (CST), and left frontopontine tract (FPT) on 36-36 (2 steps) setting. The yellow regions are labels, and the green regions are segmentation results of our method and compared methods.
• Figure 4: Quantitative comparison of Single-Label Voxel Contrast and Multi-Label Voxel Contrast of 36 tracts using DSC (%) under the 36-36 (2 steps) setting on the HCP dataset.
• Figure 5: Visualization of segmentation results of three novel tracts: Right Inferior Occipito-Frontal Fascicle (IFO right), Corpus Callosum (CC), and Right Arcuate Fascicle (AF right) on 36-36 (2 steps) setting on the Preto dataset.