The Impact of Scaling Training Data on Adversarial Robustness

TL;DR

This work investigates how training data scale and data quality shape adversarial robustness across 36 vision models spanning supervised, self-supervised, and contrastive paradigms. It demonstrates robust scaling laws: ASR scales approximately logarithmically with both data volume and model size, with a representative univariate law for data $ASR = -3.16 \log_{10}(x) + 55.53$ and for model size $ASR = -13.39 \log_{10}(x) + 141.18$, while a PCA-based bivariate law shows $ASR = -0.46 \log_{10}(x_{data}) - 12.53 \log_{10}(x_{model}) + 137.67$ and finds model size typically more impactful. Importantly, high-quality, curated data (e.g., DINOv2) can outperform much larger but less curated datasets, challenging the idea that scale alone yields robust models. Adversarial fine-tuning can generalize across geometric structure but not across color distributions, and human evaluators consistently outperform models, highlighting fundamental gaps between human and machine vision. Overall, the results emphasize that data quality, architecture, and training objectives can drive broad-spectrum adversarial resilience at least as much as raw scale, with practical implications for building robust vision systems.

Abstract

Deep neural networks remain vulnerable to adversarial examples despite advances in architectures and training paradigms. We investigate how training data characteristics affect adversarial robustness across 36 state-of-the-art vision models spanning supervised, self-supervised, and contrastive learning approaches, trained on datasets from 1.2M to 22B images. Models were evaluated under six black-box attack categories: random perturbations, two types of geometric masks, COCO object manipulations, ImageNet-C corruptions, and ImageNet-R style shifts. Robustness follows a logarithmic scaling law with both data volume and model size: a tenfold increase in data reduces attack success rate (ASR) on average by ~3.2%, whereas a tenfold increase in model size reduces ASR on average by ~13.4%. Notably, some self-supervised models trained on curated datasets, such as DINOv2, outperform others trained on much larger but less curated datasets, challenging the assumption that scale alone drives robustness. Adversarial fine-tuning of ResNet50s improves generalization across structural variations but not across color distributions. Human evaluation reveals persistent gaps between human and machine vision. These results show that while scaling improves robustness, data quality, architecture, and training objectives play a more decisive role than raw scale in achieving broad-spectrum adversarial resilience.

Figures (22)

• Figure 1: Overview of the black-box attack pipeline. Input images are modified using a semantic adversarial attack. In this example, an ImageNet image of a red fox is attacked using the GeometricMasksV2 3-4-2 C1 with an opacity of 128, causing a misclassification in the target classifier.
• Figure 2: Sample images of all attacks applied for the robustness analysis of the categories Random Perturbations, GeometricMasksV1, GeometricMasksV2, COCO-Objects, ImageNet-R, and ImageNet-C. The original image is from the ImageNet class red fox.
• Figure 3: Overall average ASR across different attack categories: Random Perturbations, GeometricMasksV1, GeometricMasksV2, COCO Objects, ImageNet-C, and ImageNet-R. The overall average is computed as the mean of the average ASR values for each attack category. We show in \ref{['fig:asr_full']} the same plot with labels for all points.
• Figure 4: Overall average ASR relative to the number of model parameters averaged across: Random Perturbation, GeometricMasksV1, GeometricMasksV2, Coco Objects, ImageNet-C, and ImageNet-R attacks. We show in \ref{['fig:asr_msize_full']} the same plot with labels for all points.
• Figure 5: Accuracies of various models and the average accuracy of human participants on the GeometricMasksV2 6-7-2 C1 mask, applied at opacities 0, 64, 96, and 128. Raw values in \ref{['tab:humans_vs_models_results']}.