GaussianImage++: Boosted Image Representation and Compression with 2D Gaussian Splatting

TL;DR

GaussianImage++ tackles the inefficiency of 2D Gaussian splatting by introducing distortion driven densification and content aware Gaussian filters to adaptively allocate and shape Gaussian primitives. It couples these enhancements with a quantization aware compression pipeline that uses learnable LSQ+ quantizers for Gaussian attributes. Experimental results on Kodak and DIV2K demonstrate improved rate-distortion performance and representation quality over GaussianImage and COIN, while preserving real time decoding and low memory usage. The work provides generalizable techniques for boosting 2D GS methods and narrows the gap with neural codecs in decoding efficiency and resource requirements.

Abstract

Implicit neural representations (INRs) have achieved remarkable success in image representation and compression, but they require substantial training time and memory. Meanwhile, recent 2D Gaussian Splatting (GS) methods (\textit{e.g.}, GaussianImage) offer promising alternatives through efficient primitive-based rendering. However, these methods require excessive Gaussian primitives to maintain high visual fidelity. To exploit the potential of GS-based approaches, we present GaussianImage++, which utilizes limited Gaussian primitives to achieve impressive representation and compression performance. Firstly, we introduce a distortion-driven densification mechanism. It progressively allocates Gaussian primitives according to signal intensity. Secondly, we employ context-aware Gaussian filters for each primitive, which assist in the densification to optimize Gaussian primitives based on varying image content. Thirdly, we integrate attribute-separated learnable scalar quantizers and quantization-aware training, enabling efficient compression of primitive attributes. Experimental results demonstrate the effectiveness of our method. In particular, GaussianImage++ outperforms GaussianImage and INRs-based COIN in representation and compression performance while maintaining real-time decoding and low memory usage.

Figures (10)

• Figure 1: Image representation based on 2D GS methods with different numbers of Gaussians. Our GaussianImage++ exhibits significant performance gains.
• Figure 2: Our proposed GaussianImage++ framework. Our representation pipeline uses densification to initialize sparse Gaussians, growing them periodically to improve under-reconstructed areas. A content-aware filter is applied to all Gaussians before accumulated sum rasterization. For compression, we initialize with overfitted 2D Gaussians and employ quantization-aware training to encode attributes into compact bitstreams.
• Figure 3: Left: the variance $s_i$ of the i-th Gaussians primitives. Right: Number of Gaussians $N_t$ at iteration $t$.
• Figure 4: Progressive training of GaussianImage++ ($M=10k$, $T=5\times 10^4$). Before densification (t=500), GaussianImage++ with CAF enables reducing the holes and artifacts and represent a coarse structure with sparse Gaussians. As densification steps (t=1000 or t=50000), the final rendered image shows enhanced visual quality.
• Figure 5: Rate-distortion curves of GaussianImage++ and different baselines on Kodak (top) and DIV2K (bottom) datasets.