Cross-Sequence Semi-Supervised Learning for Multi-Parametric MRI-Based Visual Pathway Delineation

TL;DR

This work tackles VP delineation from multi-parametric MRI under limited labeling by introducing a semi-supervised framework that couples correlation-constrained feature decomposition (CFD) with a mean-teacher–based consistency mechanism and a consistency-based sample enhancement (CSE). CFD disentangles each sequence into unique and non-unique features, guided by a Pearson-correlation loss $\mathcal{L}_{dcp}$ to capture cross-sequence dependencies, while CSE leverages unlabeled data through reliable pseudo-labels, selected via an inconsistency score and a threshold $thres=0.05$. The approach, evaluated on HCP, MDM, and multimodal MRI datasets, outperforms state-of-the-art SSL methods and remains competitive with fully supervised models when labels are scarce, achieving better VP boundary delineation and continuity. This framework reduces annotation requirements and demonstrates strong potential for clinical VP delineation, with avenues for broader validation and integration with functional analyses in the future.

Abstract

Accurately delineating the visual pathway (VP) is crucial for understanding the human visual system and diagnosing related disorders. Exploring multi-parametric MR imaging data has been identified as an important way to delineate VP. However, due to the complex cross-sequence relationships, existing methods cannot effectively model the complementary information from different MRI sequences. In addition, these existing methods heavily rely on large training data with labels, which is labor-intensive and time-consuming to obtain. In this work, we propose a novel semi-supervised multi-parametric feature decomposition framework for VP delineation. Specifically, a correlation-constrained feature decomposition (CFD) is designed to handle the complex cross-sequence relationships by capturing the unique characteristics of each MRI sequence and easing the multi-parametric information fusion process. Furthermore, a consistency-based sample enhancement (CSE) module is developed to address the limited labeled data issue, by generating and promoting meaningful edge information from unlabeled data. We validate our framework using two public datasets, and one in-house Multi-Shell Diffusion MRI (MDM) dataset. Experimental results demonstrate the superiority of our approach in terms of delineation performance when compared to seven state-of-the-art approaches.

Figures (12)

• Figure 1: t-SNE visualizations of the decomposed unique and non-unique features. 3D features were visualized by stacking the features from slices of each subject. Red: Visual pathway (VP) with T1-weight unique features; Cyan: VP with T1-weight non-unique features; Orange: Visual pathway (VP) with FA unique features; Magenta: Visual pathway (VP) with FA non-unique features; Blue: Background with T1-weight non-unique features; Purple: Background with T1-weight unique features; Green: Background with FA unique features; Navy: Background with FA non-unique features. Subplots (a) and (c) show the unique features, revealing a distinct boundary between the VP and the background. This distinct separation indicates that using unique features in the final delineation stage allows the network to accurately isolate the VP. In contrast, once the non-unique features are introduced (subplots (b) and (d)), the feature points become intertwined in the t-SNE plots, making it more challenging to distinguish between the VP and the background.
• Figure 2: The overall pipeline of the proposed framework. It consists of a CFD module to decompose MRI sequences and a CSE module to generate consistent samples from unlabeled data to augment labeled data. $\mathcal{L}_{sup}$ and $\mathcal{L}_{cons\_cse}$ are the supervised and unsupervised consistency loss, respectively. Here, we only presented the CFD module for the labeled images for simplicity, as the process is the same for unlabeled images.
• Figure 3: The detailed structure of the feature prediction block (PredNet). $C_0$, $C_1$, and $C_2$ represent the convolution layers; and $h_\lambda$ denotes the soft-thresholding operation to induce sparsity.
• Figure 4: Qualitative results comparison on the HCP dataset. In addition to the delineation results, we provide the binary masks (ground truth) for better comparison.
• Figure 5: Comparison with fully supervised methods when the available labeled data is 8, 16, and 82 samples for the HCP dataset, and 1, 3, and 8 samples for the MDM dataset.