SmartSplat: Feature-Smart Gaussians for Scalable Compression of Ultra-High-Resolution Images

TL;DR

SmartSplat tackles the challenge of ultra-high-resolution image compression by introducing a highly adaptive, feature-aware 2D Gaussian representation. It combines gradient-guided variational sampling, exclusion-based uniform sampling, and scale-adaptive color initialization to place and color Gaussians efficiently, enabling high-quality reconstruction under extreme compression with relatively few primitives. The method leverages a tile-based pipeline for scalability, a differentiable rasterizer for end-to-end optimization, and a carefully designed loss that blends $\ell_1$ distortion with SSIM-based structure preservation. Evaluations on DIV8K and the newly created DIV16K dataset show superior performance over state-of-the-art GS-based methods across a wide range of compression ratios, demonstrating strong scalability and practical applicability for real-world ultra-high-resolution content delivery.

Abstract

Recent advances in generative AI have accelerated the production of ultra-high-resolution visual content, posing significant challenges for efficient compression and real-time decoding on end-user devices. Inspired by 3D Gaussian Splatting, recent 2D Gaussian image models improve representation efficiency, yet existing methods struggle to balance compression ratio and reconstruction fidelity in ultra-high-resolution scenarios. To address this issue, we propose SmartSplat, a highly adaptive and feature-aware GS-based image compression framework that supports arbitrary image resolutions and compression ratios. SmartSplat leverages image-aware features such as gradients and color variances, introducing a Gradient-Color Guided Variational Sampling strategy together with an Exclusion-based Uniform Sampling scheme to improve the non-overlapping coverage of Gaussian primitives in pixel space. In addition, we propose a Scale-Adaptive Gaussian Color Sampling method to enhance color initialization across scales. Through joint optimization of spatial layout, scale, and color initialization, SmartSplat efficiently captures both local structures and global textures using a limited number of Gaussians, achieving high reconstruction quality under strong compression. Extensive experiments on DIV8K and a newly constructed 16K dataset demonstrate that SmartSplat consistently outperforms state-of-the-art methods at comparable compression ratios and exceeds their compression limits, showing strong scalability and practical applicability. The code is publicly available at https://github.com/lif314/SmartSplat.

Figures (15)

• Figure 1: Comparison with baselines on $4218\times 7350$ image. SmartSplat consistently outperforms baselines under the same Compression Ratio (CR) and surpasses the maximum compression limits achieved by previous approaches, maintaining high-fidelity reconstruction even at extreme compression levels (e.g., 1000×).
• Figure 2: SmartSplat maintains visual quality under extreme high compression ratios. Under maximum compression ratio ($\mathrm{CR}_{\text{max}}$), JPEG shows severe artifacts, and GI struggles with scalability. In contrast, SmartSplat outperforms GI at the same compression ratio and rivals JPEG even at 2000$\times$, maintaining visually pleasing results up to 5000$\times$. The insets visualize the corresponding error images, with brighter colors indicating higher errors.
• Figure 3: Pipeline of SmartSplat. Given an input image, SmartSplat initializes Gaussian primitives via feature-aware sampling and optimizes them through differentiable rasterization to learn compact, perceptually-aware representations.
• Figure 4: Qualitative comparison on DIV8K and DIV16K.
• Figure 5: Training convergence speed comparison.