Splatwizard: A Benchmark Toolkit for 3D Gaussian Splatting Compression

TL;DR

The paper presents Splatwizard, a unified benchmark toolkit designed to standardize the evaluation of 3D Gaussian Splatting (3DGS) compression. It offers a modular architecture with dynamic/AOT module loading, a scheduler-driven unified pipeline, and libraries for rasterization, metrics, encoding, and reconstruction, enabling reproducible reproduction of diverse 3DGS methods. A novel ChimeraGS model demonstrates how mixing rasterizers with distillation-guided pruning and structured encoding achieves competitive rate-distortion and geometric performance. Comprehensive benchmarks across six multi-view datasets assess photometric and geometric quality, rendering speed, and memory usage, highlighting both the strengths and trade-offs of current approaches and underscoring the framework’s value for fair comparisons. The framework aims to accelerate 3DGS research by providing extensibility for future tasks such as dynamic scenes and streaming, along with standardized, robust baselines for reproducibility.

Abstract

The recent advent of 3D Gaussian Splatting (3DGS) has marked a significant breakthrough in real-time novel view synthesis. However, the rapid proliferation of 3DGS-based algorithms has created a pressing need for standardized and comprehensive evaluation tools, especially for compression task. Existing benchmarks often lack the specific metrics necessary to holistically assess the unique characteristics of different methods, such as rendering speed, rate distortion trade-offs memory efficiency, and geometric accuracy. To address this gap, we introduce Splatwizard, a unified benchmark toolkit designed specifically for benchmarking 3DGS compression models. Splatwizard provides an easy-to-use framework to implement new 3DGS compression model and utilize state-of-the-art techniques proposed by previous work. Besides, an integrated pipeline that automates the calculation of key performance indicators, including image-based quality metrics, chamfer distance of reconstruct mesh, rendering frame rates, and computational resource consumption is included in the framework as well. Code is available at https://github.com/splatwizard/splatwizard

Figures (9)

• Figure 1: Framework of Splatwizard. We have abstracted the Gaussian training process into a universal training pipeline, which can support various custom operations through a task scheduler mechanism. The horizontal arrow lines in the diagram illustrate the modules used at different stages of Gaussian training. For instance, different models employ different adaptive density control mechanisms, and such control mechanisms can be registered into the scheduler. This results in a GS model implemented based on Splatwizard, while also seamlessly leveraging various auxiliary modules and functions provided by the framework.
• Figure 2: Snippet used to register task.
• Figure 3: Framework of ChimeraGS. Since Splatwizard natively supports flexible module composition, we can easily mix multiple different rasterizers in same model. The labels on the left side indicate the rasterizers used in each phase.
• Figure 4: Results of Mip-NeRF 360. Note that when counting the total number of Gaussian primitives in HAC and CAT-3DGS, only the number of anchor is considered, and the values are not directly comparable to other methods, hence they are drawn with a dashed line.
• Figure 5: Results of DTU dataset. Here we present the PSNR, along with the Chamfer distance based on mesh and point cloud respectively.