HO-FMN: Hyperparameter Optimization for Fast Minimum-Norm Attacks

TL;DR

This paper tackles unreliable adversarial robustness evaluations arising from fixed attack hyperparameters by introducing HO-FMN, a modular reimplementation of the Fast Minimum-Norm (FMN) attack that enables arbitrary differentiable losses $L$, optimizers $u$, and step-size schedulers $s$. It then applies Bayesian optimization to automatically select the best hyperparameters for each configuration, minimizing the median minimum-norm perturbation $\widetilde{\|\boldsymbol{\delta}\|}$ to rank configurations per model. Across 12 RobustBench models on CIFAR-10 and ImageNet, HO-FMN yields smaller adversarial perturbations than the FMN baseline and competitive APGD variants, while producing complete robustness evaluation curves in a single run. The approach provides a more reliable and informative assessment of model robustness, with open-source code to facilitate adoption and extension to other norms and attack variants.

Abstract

Gradient-based attacks are a primary tool to evaluate robustness of machine-learning models. However, many attacks tend to provide overly-optimistic evaluations as they use fixed loss functions, optimizers, step-size schedulers, and default hyperparameters. In this work, we tackle these limitations by proposing a parametric variation of the well-known fast minimum-norm attack algorithm, whose loss, optimizer, step-size scheduler, and hyperparameters can be dynamically adjusted. We re-evaluate 12 robust models, showing that our attack finds smaller adversarial perturbations without requiring any additional tuning. This also enables reporting adversarial robustness as a function of the perturbation budget, providing a more complete evaluation than that offered by fixed-budget attacks, while remaining efficient. We release our open-source code at https://github.com/pralab/HO-FMN.

Figures (4)

• Figure 1: Overview of our HO-FMN approach.
• Figure 2: Mean and standard deviation of the median perturbation size $\|\Tilde{\boldsymbol{\delta}}\|$ estimated by the GPR model, for a specific test configuration, as a function of the learning rate ($\gamma$) and momentum ($\mu$) hyperparameters. The pairs ($\gamma$, $\mu$) sampled during the process to iteratively refine the GPR model are shown as red points.
• Figure 3: Robustness evaluation curves for $M\xspace_1$-$M\xspace_9$. The dashed-gray and solid-blue lines represent FMN and HO-FMN. The robust accuracy (RA) value at $\epsilon = 8/255$ computed with APGD$_{\texttt{CE}\xspace/\texttt{DLR}\xspace}$ (the best value between the two) is also shown as a red cross.
• Figure 4: Robustness evaluation curves for $M\xspace_{10}$-$M\xspace_{12}$, and APGD robust accuracy at $\epsilon = 4/255$. Please refer to \ref{['fig:sec_evals_cifar10']} for further details.