ClusMFL: A Cluster-Enhanced Framework for Modality-Incomplete Multimodal Federated Learning in Brain Imaging Analysis

TL;DR

ClusMFL tackles modality incompleteness in brain-imaging multimodal federated learning by building a global pool of FINCH cluster centers per modality-label, aligning modality-specific features with supervised contrastive learning, and completing missing modalities via cluster proxies. A modality-aware aggregation strategy adaptively weighs client contributions, improving robustness as missingness increases. Across ADNI MRI and PET experiments, ClusMFL outperforms standard federated baselines and modality-specific methods, with ablations confirming the contribution of each component and faster convergence than GAN-based approaches. The approach enables robust cross-institution brain-imaging analyses under realistic incomplete-modality conditions, with potential for extension to additional modalities and efficiency enhancements.

Abstract

Multimodal Federated Learning (MFL) has emerged as a promising approach for collaboratively training multimodal models across distributed clients, particularly in healthcare domains. In the context of brain imaging analysis, modality incompleteness presents a significant challenge, where some institutions may lack specific imaging modalities (e.g., PET, MRI, or CT) due to privacy concerns, device limitations, or data availability issues. While existing work typically assumes modality completeness or oversimplifies missing-modality scenarios, we simulate a more realistic setting by considering both client-level and instance-level modality incompleteness in this study. Building on this realistic simulation, we propose ClusMFL, a novel MFL framework that leverages feature clustering for cross-institutional brain imaging analysis under modality incompleteness. Specifically, ClusMFL utilizes the FINCH algorithm to construct a pool of cluster centers for the feature embeddings of each modality-label pair, effectively capturing fine-grained data distributions. These cluster centers are then used for feature alignment within each modality through supervised contrastive learning, while also acting as proxies for missing modalities, allowing cross-modal knowledge transfer. Furthermore, ClusMFL employs a modality-aware aggregation strategy, further enhancing the model's performance in scenarios with severe modality incompleteness. We evaluate the proposed framework on the ADNI dataset, utilizing structural MRI and PET scans. Extensive experimental results demonstrate that ClusMFL achieves state-of-the-art performance compared to various baseline methods across varying levels of modality incompleteness, providing a scalable solution for cross-institutional brain imaging analysis.

Figures (3)

• Figure 1: Illustration of Modality Incompleteness Setting. Dash boxes denote the missing modality, while each pair of boxes represents an instance.
• Figure 2: Overview of ClusMFL. In this figure, PET-only instances are used as examples of single-modality instances in local training. Different patterns represent different modalities, and different colors indicate different labels.
• Figure 3: Training curves of different methods.