SINR: Sparsity Driven Compressed Implicit Neural Representations

TL;DR

SINR addresses the challenge of efficiently compressing implicit neural representations by exploiting weight-space sparsity. It represents each INR weight vector as a high-dimensional sparse code via $\mathbf{x}$ and a seed-controlled random sensing matrix $\mathbf{A}$ so that $\mathbf{w} \approx \mathbf{A}\mathbf{x}$, enabling reconstruction without transmitting a learned dictionary. The approach is compatible with existing INR compression pipelines and yields substantial reductions in storage (lower $bpp$) while preserving decoding quality across modalities such as images, occupancy fields, and NeRFs. Empirical results demonstrate that SINR provides fundamental compression by targeting the weight-space structure, offering a practical, universal, and modality-agnostic improvement over prior INR compression baselines.

Abstract

Implicit Neural Representations (INRs) are increasingly recognized as a versatile data modality for representing discretized signals, offering benefits such as infinite query resolution and reduced storage requirements. Existing signal compression approaches for INRs typically employ one of two strategies: 1. direct quantization with entropy coding of the trained INR; 2. deriving a latent code on top of the INR through a learnable transformation. Thus, their performance is heavily dependent on the quantization and entropy coding schemes employed. In this paper, we introduce SINR, an innovative compression algorithm that leverages the patterns in the vector spaces formed by weights of INRs. We compress these vector spaces using a high-dimensional sparse code within a dictionary. Further analysis reveals that the atoms of the dictionary used to generate the sparse code do not need to be learned or transmitted to successfully recover the INR weights. We demonstrate that the proposed approach can be integrated with any existing INR-based signal compression technique. Our results indicate that SINR achieves substantial reductions in storage requirements for INRs across various configurations, outperforming conventional INR-based compression baselines. Furthermore, SINR maintains high-quality decoding across diverse data modalities, including images, occupancy fields, and Neural Radiance Fields.

Figures (6)

• Figure 1: Weight distribution of INRs tends to follow a Gaussian distribution for various data modalities.
• Figure 2: The proposed SINR compression algorithm: Standard compression techniques for INRs typically involve direct quantization and entropy coding of their weights. However, since natural signals exhibit inherent compressibility in a dictionary, the characteristics that aid in the compressibility of the weight space of an INR are discovered through the Gaussian nature of the weight space. Therefore, SINR employs $L_1$ minimization to identify a higher-dimensional sparse code. Furthermore, based on the weight space observations and the CLT, we simplify the encoding and decoding process using a random sensing matrix controlled by a seed. Subsequently, only the non-zero (NZ) values and their corresponding indices are quantized and entropy coded.
• Figure 3: Experiments $C_1$ and $C_2$: Identifying compressible INR combinations. The SINR approach demonstrates that configurations in $C_1$ are more compressible than those in $C_2$. Furthermore, in both configurations SINR achieves lower bpp while maintaining the PSNR values.
• Figure 4: Experiments $C_3$, $C_4$, and $C_5$: Identifying compressible INR combinations under Meta-Learning. Meta-learning approaches have been introduced for INRs to enhance their generalization abilities and achieve faster convergence. When assessing induced sparsity in the weight space, SINR demonstrates a significant reduction in bpp values while maintaining nearly the same PSNR performance as the baselines.
• Figure 5: Results for image encoding experiment. SINR compresses the INR into a dictionary, significantly reducing the storage required compared to baseline INR image compressors. The results demonstrate that the decoded representations undergo a very negligible loss in PSNR, which is minimal considering the substantial storage space saved.