Human Aligned Compression for Robust Models

TL;DR

This paper tackles the vulnerability of vision models to adversarial perturbations by introducing human-perception-aligned learned compression as a preprocessing defense. It systematically compares JPEG with learned compressors HiFiC and ELIC on ResNet50 and Vision Transformer (ViT) architectures across Imagenette and ImageNet, including sequential compression. The results show that HiFiC and ELIC generally outperform JPEG, especially for ViT, under black-box attacks, though defenses weaken when gradients propagate through the defense (white-box). Sequential compression significantly enhances robustness with manageable computational overhead, offering a practical, perceptually aligned defense that preserves task-relevant features while removing adversarial noise.

Abstract

Adversarial attacks on image models threaten system robustness by introducing imperceptible perturbations that cause incorrect predictions. We investigate human-aligned learned lossy compression as a defense mechanism, comparing two learned models (HiFiC and ELIC) against traditional JPEG across various quality levels. Our experiments on ImageNet subsets demonstrate that learned compression methods outperform JPEG, particularly for Vision Transformer architectures, by preserving semantically meaningful content while removing adversarial noise. Even in white-box settings where attackers can access the defense, these methods maintain substantial effectiveness. We also show that sequential compression--applying rounds of compression/decompression--significantly enhances defense efficacy while maintaining classification performance. Our findings reveal that human-aligned compression provides an effective, computationally efficient defense that protects the image features most relevant to human and machine understanding. It offers a practical approach to improving model robustness against adversarial threats.

Figures (19)

• Figure 1: Visual comparison of image degradation after three compression/decompression cycles using different compression methods and quality settings. From left to right: (a) ELIC 0004, (b) ELIC quality 0016, (c) HiFiC low, (d) HiFiC medium, (e) JPEG quality 25.0, and (f) the original uncompressed image. Note how learned compression methods (ELIC, HiFiC) exhibit different artifact patterns than traditional JPEG compression.
• Figure 2: Model accuracy under FGSM (top row), iFGSM (middle row), and PGD (bottom row) adversarial attacks on the Imagenette dataset for ResNet50 (left) and ViT (right) architectures. Solid lines represent "black-box" attacks (without gradient propagation through the compression), while dashed lines show "white-box" attacks (with gradient propagation through the defense). Learned compression methods (ELIC, HiFiC) consistently outperform JPEG for the ViT model, particularly under stronger attacks. Epsilon values represent attack strength as $x/255$.
• Figure 3: Model accuracy under Carlini-Wagner (CW, top row) and DeepFool (bottom row) adversarial attacks on the Imagenette dataset for ResNet50 (left) and ViT (right) architectures. Solid lines represent "black-box" attacks, while dashed lines show "white-box" attacks. Epsilon shows the maximum L2 norm of the perturbations. HiFiC demonstrates superior robustness against CW attacks, especially for the ViT model.
• Figure 4: Model accuracy under FGSM (top row), iFGSM (middle row), and PGD (bottom row) adversarial attacks on 1000 randomly sampled images from the ImageNet dataset. Results demonstrate that the same defense patterns observed on Imagenette generalize to the more complex ImageNet classification task, though with lower overall accuracy due to the increased task difficulty.
• Figure 5: Comparison of model accuracy for different JPEG quality levels under iFGSM attacks for ResNet50 (top) and ViT (bottom). Dashed lines show results when gradient information was available to the attack ("white-box" setting). Lower quality settings provide better defense against strong attacks but reduce clean accuracy, with quality level 25.0 offering the best trade-off.