Towards Adversarially Robust Deep Metric Learning

TL;DR

This paper addresses the vulnerability of Deep Metric Learning (DML) under adversarial perturbations, particularly in clustering-based inference where existing defenses from classification do not transfer well. It introduces Ensemble Adversarial Training (EAT), which combines ensemble diversification via data splits with a self-transferring mechanism that propagates ensemble robustness statistics to individual models. The authors show that conventional defenses adapted from classification tasks underperform in DML, and demonstrate that EAT achieves superior robustness on three popular datasets (CUB200, CARS196, In-Shop) across two backbone architectures with PAL as the loss, while incurring minimal loss on clean performance. The work offers a practical methodology to enhance robust DML in clustering scenarios, improving reliability for applications such as image retrieval and person re-identification.

Abstract

Deep Metric Learning (DML) has shown remarkable successes in many domains by taking advantage of powerful deep neural networks. Deep neural networks are prone to adversarial attacks and could be easily fooled by adversarial examples. The current progress on this robustness issue is mainly about deep classification models but pays little attention to DML models. Existing works fail to thoroughly inspect the robustness of DML and neglect an important DML scenario, the clustering-based inference. In this work, we first point out the robustness issue of DML models in clustering-based inference scenarios. We find that, for the clustering-based inference, existing defenses designed DML are unable to be reused and the adaptions of defenses designed for deep classification models cannot achieve satisfactory robustness performance. To alleviate the hazard of adversarial examples, we propose a new defense, the Ensemble Adversarial Training (EAT), which exploits ensemble learning and adversarial training. EAT promotes the diversity of the ensemble, encouraging each model in the ensemble to have different robustness features, and employs a self-transferring mechanism to make full use of the robustness statistics of the whole ensemble in the update of every single model. We evaluate the EAT method on three widely-used datasets with two popular model architectures. The results show that the proposed EAT method greatly outperforms the adaptions of defenses designed for deep classification models.

Figures (4)

• Figure 1: The t-SNE Poliar731877 results of the adaptions of popular defenses (Adversarial Training madryDeepLearningModels2019a and Mix-Up zhangMixupEmpiricalRisk2018. The DML models are trained on CUB200 WelinderEtal2010 using the Proxy Anchor Loss kimProxyAnchorLoss2020. Before launching the attack, the DML model can clearly classify samples from different classes. But after the attack, DML models trained under different settings all showed poor performance.
• Figure 2: Recall@1 and F1-Score for different iteration numbers of the PGD attack with the different defenses on CUB200 (Training with MobileNetV2)
• Figure 3: Recall@1 and F1-Score for different $\varepsilon$ settings of the PGD attack with the different defenses on CUB200 (Training with the MobileNetV2)
• Figure 4: Recall@1 for the individual model of the EAT defense in comparison with other defenses on CUB200 (Training with MobileNetV2)