Towards All-in-One Medical Image Re-Identification

TL;DR

This work tackles medical image re-identification across diverse imaging modalities by introducing MaMI, an all-in-one MedReID framework. MaMI employs a Continuous Modality-based Parameter Adapter (ComPA) to dynamically convert a modality-agnostic backbone into modality-specific models at runtime, and it integrates rich medical priors from pre-trained medical foundation models through inter-image difference alignment to emphasize identity-related features. The approach is validated across 11 datasets and against numerous foundation models and large multi-modal language models, consistently outperforming baselines and demonstrating cross-modality and cross-domain generalization. Additionally, MaMI enables practical applications in history-augmented diagnosis and privacy protection by retrieving historical data and by identifying and removing identity cues from images, respectively. The work provides a thorough benchmark, detailed ablations, and publicly available code to facilitate adoption and further research in cross-modality MedReID and privacy-preserving medical imaging.

Abstract

Medical image re-identification (MedReID) is under-explored so far, despite its critical applications in personalized healthcare and privacy protection. In this paper, we introduce a thorough benchmark and a unified model for this problem. First, to handle various medical modalities, we propose a novel Continuous Modality-based Parameter Adapter (ComPA). ComPA condenses medical content into a continuous modality representation and dynamically adjusts the modality-agnostic model with modality-specific parameters at runtime. This allows a single model to adaptively learn and process diverse modality data. Furthermore, we integrate medical priors into our model by aligning it with a bag of pre-trained medical foundation models, in terms of the differential features. Compared to single-image feature, modeling the inter-image difference better fits the re-identification problem, which involves discriminating multiple images. We evaluate the proposed model against 25 foundation models and 8 large multi-modal language models across 11 image datasets, demonstrating consistently superior performance. Additionally, we deploy the proposed MedReID technique to two real-world applications, i.e., history-augmented personalized diagnosis and medical privacy protection. Codes and model is available at \href{https://github.com/tianyuan168326/All-in-One-MedReID-Pytorch}{https://github.com/tianyuan168326/All-in-One-MedReID-Pytorch}.

Figures (4)

• Figure 1: (a) We propose MaMI, an all-in-one modality-adaptive ReID model for medical images. (b) MaMI enhances personalized healthcare by integrating historical medical data. (c) MaMI detects identity cues and removes them from the original images, protecting privacy while maintaining medical utility.
• Figure 2: Overview of the proposed all-in-one MedReID framework, namely Modality-adaptive Medical Identifier (MaMI). (a) We introduce a Continuous Modality-based Parameter Adapter (ComPA) to dynamically adjust a modality-agnostic model into an input modality-specific model at runtime. (b) The adjusted model extracts the identity-related visual features from the input medical images. (c) During the optimization, we also transfer the rich medical priors from the (d) medical foundation models (MFMs) to the MedReID task, by aligning the inter-image key differences. We illustrate with X-ray images, though our method also supports other modalities.
• Figure 3: Impact of the historical image number on diagnosis outcome. We use the proposed MaMI to collect the historical image, as the auxiliary information, to aid the diagnosis.
• Figure 4: Left: t-SNE map of the learned continuous modality. Right: Impact of the code number of the codebook within ComPA.