GaussianImage: 1000 FPS Image Representation and Compression by 2D Gaussian Splatting

TL;DR

GaussianImage introduces an explicit, 2D Gaussian–based image representation to overcome memory and decoding bottlenecks of implicit neural representations. It employs 2D Gaussians parameterized by μ, Σ (via factorized forms such as Σ = LL^⊤ or Σ = (RS)(RS)^⊤), plus color and opacity, and renders via an accumulated summation C_i = ∑_n c′_n exp(−σ_n) with σ_n = 1/2 d_n^⊤ Σ_n^{−1} d_n, removing depth ordering. A two‑stage compression pipeline uses 16‑bit positions, 6‑bit covariance quantization, and RVQ for color with M = 2 stages, plus optional partial bits‑back coding, yielding rate–distortion comparable to COIN/COIN++ and decoding speeds ~2000 FPS. Comprehensive ablations show the benefits of accumulated blending, 2D versus 3D Gaussians, and the chosen quantization/regularization strategies, underscoring the method’s practical efficiency and potential for neural image codecs and downstream vision tasks. This work paves the way for efficient, explicit image representations suitable for resource‑constrained devices and fast, scalable neural coding.

Abstract

Implicit neural representations (INRs) recently achieved great success in image representation and compression, offering high visual quality and fast rendering speeds with 10-1000 FPS, assuming sufficient GPU resources are available. However, this requirement often hinders their use on low-end devices with limited memory. In response, we propose a groundbreaking paradigm of image representation and compression by 2D Gaussian Splatting, named GaussianImage. We first introduce 2D Gaussian to represent the image, where each Gaussian has 8 parameters including position, covariance and color. Subsequently, we unveil a novel rendering algorithm based on accumulated summation. Remarkably, our method with a minimum of 3$\times$ lower GPU memory usage and 5$\times$ faster fitting time not only rivals INRs (e.g., WIRE, I-NGP) in representation performance, but also delivers a faster rendering speed of 1500-2000 FPS regardless of parameter size. Furthermore, we integrate existing vector quantization technique to build an image codec. Experimental results demonstrate that our codec attains rate-distortion performance comparable to compression-based INRs such as COIN and COIN++, while facilitating decoding speeds of approximately 2000 FPS. Additionally, preliminary proof of concept shows that our codec surpasses COIN and COIN++ in performance when using partial bits-back coding. Code is available at https://github.com/Xinjie-Q/GaussianImage.

Figures (8)

• Figure 1: Image representation (left) and compression (right) results with different decoding time on the Kodak and DIV2K dataset, respectively. The radius of each point indicates the parameter size (left) or bits per pixel (right). Our method enjoys the fastest decoding speed regardless of parameter size or bpp.
• Figure 2: Our proposed GaussianImage framework. 2D Gaussians are first formatted and then rasterized to generate the output image. The rasterizer uses our proposed accumulated blending for efficient 2D image representation.
• Figure 3: Compression pipeline of our proposed GaussianImage. After overfitting image, we apply attribute quantization-aware fine-tuning to build an ultra-fast image codec. Partial bits-back coding is used to achieve the best compression performance.
• Figure 4: Rate-distortion curves of our approach and different baselines on Kodak and DIV2K datasets in PSNR and MS-SSIM. BB denotes partial bits-back coding. Bound denotes the theoretical rate of our codec.
• Figure 5: Subjective comparison of various codecs on Kodak at low Bpp.