Revisiting the Relation Between Robustness and Universality

TL;DR

This work reexamines the modified universality hypothesis (MUH) which posits that adversarially robust models trained for a task learn highly similar representations. Through extensive experiments across multiple datasets and a suite of representational and predictive similarity measures, the authors find that predictive universality does not hold: robust models do not converge to identical predictions on regular data as robustness increases. Representations, however, become more similar under robustness in some settings (notably ImageNet1k), though the degree of similarity is dataset-dependent and available measures may give conflicting impressions. Importantly, the study shows that the classifier at the end of the network largely drives predictive differences, and retraining the classifier can push predictions toward universality, indicating partial universality—robustness enforces some universal properties but not a blanket, universal behavior across all components.

Abstract

The modified universality hypothesis proposed by Jones et al. (2022) suggests that adversarially robust models trained for a given task are highly similar. We revisit the hypothesis and test its generality. While we verify Jones' main claim of high representational similarity in specific settings, results are not consistent across different datasets. We also discover that predictive behavior does not converge with increasing robustness and thus is not universal. We find that differing predictions originate in the classification layer, but show that more universal predictive behavior can be achieved with simple retraining of the classifiers. Overall, our work points towards partial universality of neural networks in specific settings and away from notions of strict universality.

Figures (8)

• Figure 1: Predictive behavior remains distinct at high robustness contrary to the MUH. Distributions of agreement (A) and scaled Jensen-Shannon divergence (JSDSim) (B) across all model pairs when given regular images. The MUH predicts that robust models converge to a universal solution, which should be reflected in highly similar predictions with increasing $L_2$ robustness $\epsilon$. However, predictions do not converge with increasing robustness, with agreement dropping and JSD showing increasing variance. This points towards an issue with the MUH.
• Figure 2: Image inversion algorithm and examples of inverted images. (Left) Inverted images are created by iteratively updating the seed image such that its representation becomes similar to that of the target image. It aims to introduce just the relevant features for the model and reduce feature cooccurrence. (Right) Examples of inverted images for VGG-16 and DenseNet-161 trained on ImageNet1k given the seed and target images shown on the left. The top row shows results for standard models ($\epsilon=0)$, the bottom one for robust models ($\epsilon=3$). The inverted images produced by robust and standard models are quite different. Inverted images of standard models are visually extremely similar to the seed image. For robust models, inverted images contain elements clearly belonging to the target image. They show how feature cooccurrence can be lessened, e.g., the dark background of the fish was not added to the image as both robust models mainly rely on the fins and texture of the fish.
• Figure 3: Similarity of predictions on inverted images increases with robustness. Agreement (A) and JSDSim (B) distributions across all model pairs when given inverted images. Both agreement and JSDSim increase with increasing robustness on both ImageNet1k and CIFAR-10. This means that robustness does lead to increased similarity in some aspects of the models, but arguably not to universality as the absolute similarity values still reveal differences between models.
• Figure 4: Regular and mechanistic representational similarity over multiple similarity measures and datasets. Contrary to recent work, multiple similarity measures generally agree on a positive correlation between robustness and similarity. However, results from ImageNet1k do not generalize to ImageNet100 and CIFAR-10 for regular similarity (top row), which lessens the generality of the MUH. For these two datasets, only Jaccard scores have significant Spearman rank correlation with robustness. At the same time, similarity is high on an absolute level for ImageNet100, but not for CIFAR-10. While results are consistent for mechanistic similarity (bottom row), robustness alone does not lead to universality.
• Figure 5: Representational similarity of instances with agreeing or disagreeing predictions for ImageNet1k. For the global representational similarity measures, representations of instances with disagreeing predictions are surprisingly more similar than those of agreeing predictions. Only the local Jaccard similarity ($k=10$) assigns slightly lower similarity to disagreeing instances. Hence, to explain the increasing disagreement with robustness, other parts of the models have to diverge as representations consistently become more similar with increased robustness.