MUGSQA: Novel Multi-Uncertainty-Based Gaussian Splatting Quality Assessment Method, Dataset, and Benchmarks

TL;DR

This work addresses the perceptual quality evaluation gap for Gaussian Splatting (GS) reconstructions under input uncertainties by introducing a unified multi-distance subjective quality assessment (SQA) framework and the large-scale MUGSQA dataset. It combines synthetic, uncertainty-rich data generation with a crowdsourced subjective study across multiple viewing distances, enabling robust MOS estimates and two benchmarks: GS-based reconstruction robustness and metric evaluation. Analyses show current 2D IQA metrics inadequately reflect GS distortions, while certain GS methods (e.g., Mip-Splatting) demonstrate superior robustness; the study highlights the need for developing GS-specific quality metrics and distortion analyses. Overall, MUGSQA and the accompanying benchmarks provide a path toward standardized GSQA assessment with practical implications for fair comparison and method improvement in GS-based 3D reconstruction, with dataset and code to be released soon.

Abstract

Gaussian Splatting (GS) has recently emerged as a promising technique for 3D object reconstruction, delivering high-quality rendering results with significantly improved reconstruction speed. As variants continue to appear, assessing the perceptual quality of 3D objects reconstructed with different GS-based methods remains an open challenge. To address this issue, we first propose a unified multi-distance subjective quality assessment method that closely mimics human viewing behavior for objects reconstructed with GS-based methods in actual applications, thereby better collecting perceptual experiences. Based on it, we also construct a novel GS quality assessment dataset named MUGSQA, which is constructed considering multiple uncertainties of the input data. These uncertainties include the quantity and resolution of input views, the view distance, and the accuracy of the initial point cloud. Moreover, we construct two benchmarks: one to evaluate the robustness of various GS-based reconstruction methods under multiple uncertainties, and the other to evaluate the performance of existing quality assessment metrics. Our dataset and benchmark code will be released soon.

Figures (3)

• Figure 1: MUGSQA. In Step 1, we select 55 source models, render, and sample on them. During this process, we simulate a total of 54 combinations of uncertainties that might cause differences. In Step 2, we first employ 6 GS-based methods to reconstruct these models. Then, we render all samples and their source models into videos and filter them according to their quality. In Step 3, we utilize these videos and our SQA method to collect quality scores during subjective experiments. In Step 4, we filter the scores and complete the dataset. Finally, we construct two benchmarks aimed at evaluating existing metrics and comparing the robustness of different GS-based reconstruction methods.
• Figure 2: Data Generation Pipeline. From left to right, the first part represents the process of generating distorted samples and SQA videos; the second and third parts represent the reconstruction input uncertainty rendering settings and the rendering settings for SQA videos in Blender, respectively. The "Share" in the figure indicates the use of the same camera parameters. The "Reconstruction*" and "Splatting*" steps in the figure represent the use of the corresponding algorithm based on the selected GS-based reconstruction method.
• Figure 3: Scoring Interface.