Hierarchical Self-Supervised Adversarial Training for Robust Vision Models in Histopathology

TL;DR

This work addresses the vulnerability of vision models in histopathology to adversarial perturbations. It introduces Hierarchical Self-Supervised Adversarial Training (HSAT), a min–max framework that exploits the patient–slide–patch hierarchy via a hierarchical contrastive objective and hierarchy-wise perturbations crafted with PGD. On the OpenSRH dataset, HSAT delivers substantial white-box robustness gains and reduces transfer-based black-box degradation compared to prior SSL and instance-level defenses, establishing a new robustness benchmark for biomedical imaging. The approach promises practical impact by yielding more reliable histopathology models and provides open-source code for replication and broader adoption.

Abstract

Adversarial attacks pose significant challenges for vision models in critical fields like healthcare, where reliability is essential. Although adversarial training has been well studied in natural images, its application to biomedical and microscopy data remains limited. Existing self-supervised adversarial training methods overlook the hierarchical structure of histopathology images, where patient-slide-patch relationships provide valuable discriminative signals. To address this, we propose Hierarchical Self-Supervised Adversarial Training (HSAT), which exploits these properties to craft adversarial examples using multi-level contrastive learning and integrate it into adversarial training for enhanced robustness. We evaluate HSAT on multiclass histopathology dataset OpenSRH and the results show that HSAT outperforms existing methods from both biomedical and natural image domains. HSAT enhances robustness, achieving an average gain of 54.31% in the white-box setting and reducing performance drops to 3-4% in the black-box setting, compared to 25-30% for the baseline. These results set a new benchmark for adversarial training in this domain, paving the way for more robust models. Our Code for training and evaluation is available at https://github.com/HashmatShadab/HSAT.

Figures (1)

• Figure 1: HSAT Framework: Adversarial examples are generated by maximizing the distance between images and their corresponding positive pairs across patch, slide, and patient (maximization). The model is then updated by minimizing the hierarchical loss on adversarial images (minimization), enforcing robust feature learning at all levels.