ROFI: A Deep Learning-Based Ophthalmic Sign-Preserving and Reversible Patient Face Anonymizer

TL;DR

This work introduces ROFI, a deep learning-based framework for ophthalmology that anonymizes patient faces while preserving disease-related ophthalmic signs. It combines a weakly supervised Ophthalmic Sign Detector with a Transformer-based Neural Identity Translator and a DA-Former for artifact refinement, enabling reversible identity restoration via a private key. Across three clinical centers and eleven eye-disease diagnoses, ROFI achieves strong privacy protection (over 95% anonymized images, $\kappa>0.90$ for signs) while maintaining diagnostic accuracy (health-state sensitivity 100% and $\kappa\geq0.81$ for diseases) and demonstrating high compatibility with medical AI models ($AUROC\approx0.91$). The framework also supports robust reversibility for audits and longitudinal care, with strong empirical privacy against face recognition and resilience to attacks, highlighting its potential to enable privacy-safe digital ophthalmology at scale.

Abstract

Patient face images provide a convenient mean for evaluating eye diseases, while also raising privacy concerns. Here, we introduce ROFI, a deep learning-based privacy protection framework for ophthalmology. Using weakly supervised learning and neural identity translation, ROFI anonymizes facial features while retaining disease features (over 98\% accuracy, $κ> 0.90$). It achieves 100\% diagnostic sensitivity and high agreement ($κ> 0.90$) across eleven eye diseases in three cohorts, anonymizing over 95\% of images. ROFI works with AI systems, maintaining original diagnoses ($κ> 0.80$), and supports secure image reversal (over 98\% similarity), enabling audits and long-term care. These results show ROFI's effectiveness of protecting patient privacy in the digital medicine era.

Figures (9)

• Figure 1: Study Overview.a) ROFI safeguards patient privacy while preserving eye disease-related signs, by introducing two data-driven deep neural networks: the Ophthalmic Sign Detector and the Neural Identity Protector. The Ophthalmic Sign Detector identifies and retains ophthalmic signs of the original image, while the Neural Identity Protector alters bio-identifying facial information. These features are combined and refined by the DA-Former to produce final high-quality images that are both privacy-preserving and clinically usable. The protected images can be readily diagnosed by human physicians and medical AIs. Diagnostic consistency is measured using Cohen's Kappa value ($\kappa$), where $k > 0.81$ indicates remarkable agreement. b) ROFI supports confidential reversibility with another deep network Neural Identity Restorer, enabling the accurate reconstruction of original images, using a private key established during the protection process. The reversibility is critical for medical audits, ensuring that patient information can be traced-back and accurately recorded. Moreover, this facilitates personalized medical record retrieval, supporting longitudinal evaluations, such as the assessment of thyroid eye disease (TED) therapy outcomes. c) Cohort building. The developing set is used for developing the ROFI models, and a model-selection set is adopted for selecting the best model. One internal and two external validation sets are built to evaluate the selected ROFI model. SNPH: Shanghai Ninth People's Hospital; ECXHCSU: Eye Center of Xiangya Hospital of Central South University; WURH: Renmin Hospital of Wuhan University. Icons are from https://uxwing.com/ and https://www.biorender.com/. Created with BioRender.com.
• Figure 2: Eye Sign-Level Evaluation.a) Workflow of the eye sign evaluation procedure. Eye keypoints were obtained from both original and protected images. The Mean Squared Errors (MSEs) of these keypoints were then calculated. Additionally, physicians annotated four common ophthalmic signs, namely, abnormal conjunctiva, abnormal eyelid, abnormal sclera, and abnormal iris, for both original and protected images. Then, the result consistency was assessed. b) Keypoint measurements on the SNPH validation set in terms of eyelid and iris error. Results on the ECXHCSU and WUPH validation sets are detailed in the Supplementary Figure 1. The mean and the standard deviation (SD) values are also provided in the Supplementary Table 1. Per-disease results are provided in Supplementary Figure 3 and Supplementary Figure 4. c) Illustration of the eye signs evaluated in our study. d) Eye sign consistency (Cohen's $\kappa$) on the SNPH, ECXHCSU, and WUPH validation sets. $k \textgreater 0.81$ indicates remarkable consistency dettori2020kappa. $P$ values between our result and the results achieved by other approaches were calculated with the two-sided McNemar's test pembury2020effective. Detailed results and 95% Confidence Intervals (CIs) are provided in Supplementary Table 3. Icons are from https://uxwing.com/ and https://www.biorender.com/. Created with BioRender.com. Conj-Dis:conjunctival disorder; LA-Dis: lacrimal apparatus disorder; Eyelid-Dis: eyelid disorder; Iris-Dis: iris disorder. e, Qualitative comparison of the eye signs of the images processed by different privacy protection methods.
• Figure 3: Eye Disease-Level Evaluation.a) Workflow of the diagnostic evaluation procedure. Three ophthalmologists first judged whether each image is with disease or not (Yes-or-No). The final result was determined by majority voting. The same three physicians then diagnosed the specific disease by selecting from a predefined list. If more than two physicians provided the same result, that result was chosen. If all three physicians provided different results, the final result was determined through a consultation. The above procedure was repeated for all privacy protection methods and all validation sets. Finally, the results were analyzed by a statistician. b) Scatter plot indicating the sensitivity of the diagnosis results on images protected by different methods. Details are provided in Supplementary Table 4. c) Radar plots illustrating diagnostic consistency (Cohen's $\kappa$) for the specified ophthalmic diseases. Curves of varying colors represent different privacy protection methods. A value of $\kappa \geq 0.81$ indicates excellent diagnostic consistency dettori2020kappa between the original and privacy-protected images. Detailed results and 95% Confidence Intervals (CIs) of $\kappa$ are provided in Supplementary Table 5. The Matthews Correlation Coefficient (MCC) chicco2020advantages comparison of different methods are provided in Supplementary Table 6 and Supplementary Figure 4. We also report BCC: basal cell carcinoma; SCC: squamous cell carcinoma; CM: conjunctival melanoma; CL: corneal leukoma; TED: thyroid eye disease; EoE: entropion or ectropion; MN: melanocytic nevi; DEN: divided eyelid nevus. d) Confusion matrix comparison of different methods on WUPH. The confusion matrices on SNPH and ECXHCSU are provided in Supplementary Figure 5 and Supplementary Figure 6. Icons are from https://uxwing.com/ and https://www.biorender.com/. Created with BioRender.com.
• Figure 4: Compatibility to AI Diagnostic Models.a) Diagnosis consistency between protected and original images, using ResNet50 diagnostic model. b) Diagnosis consistency, using ViT diagnostic model. c) Area Under the Receiver Operating Characteristic Curve (AUROC) comparison. Models were trained with five distinct random seeds and evaluated on the test set to produce five replicates. To assess whether ROFI's performance superiority over the second-best method is significant, we calculated the $P$ value using a two-sided t-test. d) ROC curves of the ViT diagnostic model on SNPH for images protected by different privacy protection methods. The ROC curves for the ViT diagnostic model on ECXHCSU and the ResNet50 diagnostic models on both SNPH and ECXHCSU are provided in the Supplementary Figure 7, Supplementary Figure 8 and Supplementary Figure 9.
• Figure 5: Face Protection and Reversible Capability.a) Workflow for evaluating the privacy protection and the reversible reconstruction. b) ID protection rate. A higher value indicates a greater proportion of images successfully protected. We use two prominent face recognition methods, AdaCos and ArcFace. c) The conventional eye-cropping method is ineffective for privacy protection. Even a single cropped eye can lead to successful patient identification. d) Quantitative assessment of reversibly reconstructed image quality. e) TED hormone treatment efficacy, by comparing the current image with the retrieved history image. f) Qualitative comparison of the reconstructed images.