Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NIC-RobustBench: A Comprehensive Open-Source Toolkit for Neural Image Compression and Robustness Analysis

Georgii Bychkov, Khaled Abud, Egor Kovalev, Alexander Gushchin, Sergey Lavrushkin, Dmitriy Vatolin, Anastasia Antsiferova

TL;DR

NIC-RobustBench is introduced, an open-source benchmark and evaluation framework for adversarial robustness of NIC methods, and provides a broad empirical study of modern NICs and defenses in adversarial scenarios, highlighting failure modes, least and most resilient architectures, and other insights into NIC robustness.

Abstract

Neural image compression (NIC) is increasingly used in computer vision pipelines, as learning-based models are able to surpass traditional algorithms in compression efficiency. However, learned codecs can be unstable and vulnerable to adversarial attacks: small perturbations may cause severe reconstruction artifacts or indirectly break downstream models. Despite these risks, most NIC benchmarks only emphasize rate-distortion (RD) performance, focusing on model efficiency in safe, non-adversarial scenarios, while NIC robustness studies cover only specific codecs and attacks. To fill this gap, we introduce \textbf{NIC-RobustBench}, an open-source benchmark and evaluation framework for adversarial robustness of NIC methods. The benchmark integrates 8 attacks, 9 defense strategies, standard RD metrics, a large and extensible set of codecs, and tools for assessing both the robustness of the compression model and impact on downstream tasks. Using NIC-RobustBench, we provide a broad empirical study of modern NICs and defenses in adversarial scenarios, highlighting failure modes, least and most resilient architectures, and other insights into NIC robustness. Our code is available online at https://github.com/msu-video-group/NIC-RobustBench.

NIC-RobustBench: A Comprehensive Open-Source Toolkit for Neural Image Compression and Robustness Analysis

TL;DR

NIC-RobustBench is introduced, an open-source benchmark and evaluation framework for adversarial robustness of NIC methods, and provides a broad empirical study of modern NICs and defenses in adversarial scenarios, highlighting failure modes, least and most resilient architectures, and other insights into NIC robustness.

Abstract

Neural image compression (NIC) is increasingly used in computer vision pipelines, as learning-based models are able to surpass traditional algorithms in compression efficiency. However, learned codecs can be unstable and vulnerable to adversarial attacks: small perturbations may cause severe reconstruction artifacts or indirectly break downstream models. Despite these risks, most NIC benchmarks only emphasize rate-distortion (RD) performance, focusing on model efficiency in safe, non-adversarial scenarios, while NIC robustness studies cover only specific codecs and attacks. To fill this gap, we introduce \textbf{NIC-RobustBench}, an open-source benchmark and evaluation framework for adversarial robustness of NIC methods. The benchmark integrates 8 attacks, 9 defense strategies, standard RD metrics, a large and extensible set of codecs, and tools for assessing both the robustness of the compression model and impact on downstream tasks. Using NIC-RobustBench, we provide a broad empirical study of modern NICs and defenses in adversarial scenarios, highlighting failure modes, least and most resilient architectures, and other insights into NIC robustness. Our code is available online at https://github.com/msu-video-group/NIC-RobustBench.

Paper Structure

This paper contains 26 sections, 7 equations, 17 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Formulation
  4. NIC-RobustBench Framework
  5. Framework architecture.
  6. NIC Robustness Benchmark
  7. Evaluation Results
  8. NIC performance evaluation
  9. Attack efficiency analysis
  10. Defense evaluation
  11. Additional results
  12. Conclusion
  13. Appendix
  14. Contents
  15. Related NIC works
  16. ...and 11 more sections

Figures (17)

  • Figure 1: Adversarial attack on NIC with possible attack goals.
  • Figure 2: Examples of natural and artificial adversarial examples for neural compression (JPEG AI model). The left image is taken from the Open Images dataset kuznetsova2020open, the right is taken from the Kodak dataset kodakDataset.
  • Figure 3: Summary of the contents of our framework.
  • Figure 4: Overview of NIC-RobustBench modular framework structure and NIC evaluation pipeline.
  • Figure 5: Robustness evaluation of NIC models measured across (a) different datasets, (b) different attacks, and (c) different objectives.
  • ...and 12 more figures