A-QCF-Net: An Adaptive Quaternion Cross-Fusion Network for Multimodal Liver Tumor Segmentation from Unpaired Datasets

TL;DR

A-QCF-Net introduces a dual-stream quaternion network that learns a unified liver tumor segmentation model from completely unpaired CT and MRI datasets by employing Adaptive Quaternion Cross-Fusion blocks for bidirectional knowledge transfer. The architecture enforces a shared, modality-agnostic feature space via a shared quaternion bottleneck, enabling robust cross-modal regularization without synthetic image generation. Empirical results on LiTS and ATLAS show state-of-the-art Dice scores and favorable boundary metrics, with strong generalization and an accompanying explainability analysis using Grad-CAM++ supporting clinical relevance. A radiologist reader study confirms high segmentation quality and clinical acceptability, illustrating the practical potential of unpaired cross-modal learning for leveraging large unpaired clinical archives.

Abstract

Multimodal medical imaging provides complementary information that is crucial for accurate delineation of pathology, but the development of deep learning models is limited by the scarcity of large datasets in which different modalities are paired and spatially aligned. This paper addresses this fundamental limitation by proposing an Adaptive Quaternion Cross-Fusion Network (A-QCF-Net) that learns a single unified segmentation model from completely separate and unpaired CT and MRI cohorts. The architecture exploits the parameter efficiency and expressive power of Quaternion Neural Networks to construct a shared feature space. At its core is the Adaptive Quaternion Cross-Fusion (A-QCF) block, a data driven attention module that enables bidirectional knowledge transfer between the two streams. By learning to modulate the flow of information dynamically, the A-QCF block allows the network to exchange abstract modality specific expertise, such as the sharp anatomical boundary information available in CT and the subtle soft tissue contrast provided by MRI. This mutual exchange regularizes and enriches the feature representations of both streams. We validate the framework by jointly training a single model on the unpaired LiTS (CT) and ATLAS (MRI) datasets. The jointly trained model achieves Tumor Dice scores of 76.7% on CT and 78.3% on MRI, significantly exceeding the strong unimodal nnU-Net baseline by margins of 5.4% and 4.7% respectively. Furthermore, comprehensive explainability analysis using Grad-CAM and Grad-CAM++ confirms that the model correctly focuses on relevant pathological structures, ensuring the learned representations are clinically meaningful. This provides a robust and clinically viable paradigm for unlocking the large unpaired imaging archives that are common in healthcare.

Figures (8)

• Figure 1: The conceptual framework of our unpaired cross-modal learning approach. Data from separate, unpaired CT and MRI cohorts are processed in parallel. An abstract knowledge exchange mechanism allows the network to transfer generalized features (e.g., sharp boundary information from CT, soft-tissue texture from MRI) between the streams. This enriches the segmentation capability of both modalities, leading to improved tumor delineation and more accurate annotations from a single, jointly-trained model.
• Figure 2: The detailed architecture of the A-QCF-Net. Unpaired CT and MRI inputs are processed by parallel quaternion encoders. At each scale, the Adaptive QCF (A-QCF) blocks (detailed in Fig. \ref{['fig:aqcf_diagram']}) facilitate dynamic knowledge transfer. Both streams converge into a shared quaternion bottleneck to learn a common feature representation. The decoders use attention-gated (AG) skip connections to reconstruct the segmentation outputs. A joint loss drives the optimization.
• Figure 3: Detailed visualization of the Adaptive Quaternion Cross-Fusion (A-QCF) block for the MRI-to-CT knowledge transfer path. The block generates a context vector from the MRI stream, modulates it with a data-driven adaptive gate ($\lambda$), and fuses the resulting gated context with the CT feature map to produce an enriched representation ($X'_{ct}$). The process occurs symmetrically for the CT-to-MRI path.
• Figure 4: Training and validation curves for A-QCF-Net, averaged over the five cross-validation folds. (a) Joint Dice-CE loss and (b) Mean Dice Score. The stable convergence and minimal gap between training and validation curves demonstrate a robust training process free of significant overfitting.
• Figure 5: CT qualitative comparison across multiple axial slices. Columns: Ground Truth, A-QCF-Net (Ours), FRA-UNet + CRF, Attention U-Net, Swin UNETR, ResTransUnet, Diff4MMLITS, and Unpaired via KD. Overlays: liver (red) and tumor (blue).