MedBlindTuner: Towards Privacy-preserving Fine-tuning on Biomedical Images with Transformers and Fully Homomorphic Encryption

TL;DR

MedBlindTuner tackles privacy concerns in outsourced medical-imaging ML by enabling fine-tuning on CKKS-based FHE-encrypted data using a data-efficient image transformer (DEiT). The framework splits work between a hospital (feature extraction on plaintext, encryption of features) and a cloud (encrypted fine-tuning using Nesterov gradient and encrypted matrix multiplication), with decryption confined to the hospital. Experiments on five MedMNIST v2 datasets show encrypted training achieving accuracy within 1–2 percentage points of plaintext baselines, albeit with a substantial runtime overhead (approximately $30\times$ longer). The work demonstrates the practicality of privacy-preserving cloud-based ML for healthcare and outlines future work to scale to more complex datasets and optimize performance.

Abstract

Advancements in machine learning (ML) have significantly revolutionized medical image analysis, prompting hospitals to rely on external ML services. However, the exchange of sensitive patient data, such as chest X-rays, poses inherent privacy risks when shared with third parties. Addressing this concern, we propose MedBlindTuner, a privacy-preserving framework leveraging fully homomorphic encryption (FHE) and a data-efficient image transformer (DEiT). MedBlindTuner enables the training of ML models exclusively on FHE-encrypted medical images. Our experimental evaluation demonstrates that MedBlindTuner achieves comparable accuracy to models trained on non-encrypted images, offering a secure solution for outsourcing ML computations while preserving patient data privacy. To the best of our knowledge, this is the first work that uses data-efficient image transformers and fully homomorphic encryption in this domain.

Figures (2)

• Figure 1: Overview of end-to-end MedBlindTuner
• Figure 2: MedBlindTuner performance on various datasets