Extreme Compression of Adaptive Neural Images

TL;DR

This work investigates extreme compression of images represented as implicit neural representations (INRs) by introducing Adaptive Neural Images (ANI). ANI combines neural-architecture search (via Once-for-All) with quantization-aware training to produce a single, adaptable network that can be truncated into sub-networks for different memory and bandwidth constraints, achieving up to an eightfold reduction in bits-per-pixel ($bpp$) while preserving fidelity. It demonstrates that 4-bit quantization with QAT delivers state-of-the-art PSNR/$bpp$ on Kodak and extends to NeRF-style 3D representations with significant size reductions and minimal perceptual loss. The proposed framework provides a practical, transferable approach for compressed neural fields, with potential applications in streaming, remote sensing, and other bandwidth-constrained scenarios.

Abstract

Implicit Neural Representations (INRs) and Neural Fields are a novel paradigm for signal representation, from images and audio to 3D scenes and videos. The fundamental idea is to represent a signal as a continuous and differentiable neural network. This new approach poses new theoretical questions and challenges. Considering a neural image as a 2D image represented as a neural network, we aim to explore novel neural image compression. In this work, we present a novel analysis on compressing neural fields, with focus on images and introduce Adaptive Neural Images (ANI), an efficient neural representation that enables adaptation to different inference or transmission requirements. Our proposed method allows us to reduce the bits-per-pixel (bpp) of the neural image by 8 times, without losing sensitive details or harming fidelity. Our work offers a new framework for developing compressed neural fields. We achieve a new state-of-the-art in terms of PSNR/bpp trade-off thanks to our successful implementation of 4-bit neural representations.

Figures (11)

• Figure 1: Comparison with traditional codecs. Our proposed neural image ANI (at 4-bits) state-of-the-art, high-fidelity results without clearly unpleasant artifacts. Note that all the images are $\approx0.3$ bpp. Images taken from the Kodak dataset IDs 13 and 24.
• Figure 2: We illustrate the general concepts around neural image representations tancik2020fouriersitzmann2020implicit. INRs can be generalized to other sorts of signals such as audio or 3D representations.
• Figure 3: We illustrate the general concepts around image compression and transmission using INRs strumpler2022implicit. Our approach enables to adapt to diverse scenarios depending on the bandwidth, memory, and target device requirements.
• Figure 4: Comparison between PTQ and QAT. Visual results on Kodak kodak at different bit-widths. We can appreciate how at 4-bits PTQ loses the signal, while QAT maintains high fidelity. Our method improves previous approaches strumpler2022implicitdupont2021coindupont2022coin++.
• Figure 5: Comparison of our approach on the Kodak dataset with other methods. We achieved state-of-the-art performance, surpassing even newer methods such as SHACIRA girish2023shacira. Note that ANI-MFN is a single neural network that can be adapted to different bpp requirements, unlike Coin dupont2021coindupont2022coin++ or SIREN sitzmann2020implicitstrumpler2022implicit.