Arbitrary-Scale 3D Gaussian Super-Resolution

TL;DR

This work tackles arbitrary-scale 3D Gaussian super-resolution for high-fidelity novel view synthesis under resource constraints. It introduces a unified framework with three key components: scale-aware rendering to prevent aliasing across scales, generative prior-guided optimization using latent distillation sampling and orthogonal refinement to supervise fine details without ground-truth HR views, and progressive super-resolving to maintain structural consistency as scale increases. The approach yields substantial PSNR/SSIM improvements and favorable perceptual quality (low FID/LPIPS) across four benchmarks, including non-integer scales, while achieving real-time rendering at 85 FPS on 1080p. These results demonstrate that a single 3D Gaussian model can support continuous, high-quality HR rendering at arbitrary scales, enabling practical, resource-efficient HRNVS applications.

Abstract

Existing 3D Gaussian Splatting (3DGS) super-resolution methods typically perform high-resolution (HR) rendering of fixed scale factors, making them impractical for resource-limited scenarios. Directly rendering arbitrary-scale HR views with vanilla 3DGS introduces aliasing artifacts due to the lack of scale-aware rendering ability, while adding a post-processing upsampler for 3DGS complicates the framework and reduces rendering efficiency. To tackle these issues, we build an integrated framework that incorporates scale-aware rendering, generative prior-guided optimization, and progressive super-resolving to enable 3D Gaussian super-resolution of arbitrary scale factors with a single 3D model. Notably, our approach supports both integer and non-integer scale rendering to provide more flexibility. Extensive experiments demonstrate the effectiveness of our model in rendering high-quality arbitrary-scale HR views (6.59 dB PSNR gain over 3DGS) with a single model. It preserves structural consistency with LR views and across different scales, while maintaining real-time rendering speed (85 FPS at 1080p).

Figures (9)

• Figure 1: Visual results of typical solutions for arbitrary-scale 3D Gaussian super-resolution. Continuously rendering high-resolution novel views of different scale factors with vanilla 3DGS leads to aliasing artifacts. Cascaded solutions produce altered contents (e.g., StableSR). Anti-aliasing Mip-Splatting and GaussianSR yield limited details.
• Figure 2: (a) Accurate pixel shading requires aligning the integration window with the actual pixel size. (b) Approximation error analysis regarding the window size, where the proposed method results in low approximation errors.
• Figure 3: Generative prior-guided optimization, where generative priors are leveraged to constrain details in rendered HR views. To alleviate view inconsistency introduced by generative priors, optimization is conducted in the latent space, and texture supervision is applied only to orthogonal views.
• Figure 4: Progressive super-resolving. The training process is divided into multiple stages, with each stage following the same mechanism while progressively rendering higher-resolution views. Structural loss is applied between adjacent stages to ensure consistency across scale factors.
• Figure 5: Qualitative comparisons on the Mip-NeRF360 dataset, where Mip and Analytic denote Mip-Splatting and Analytic-Splatting, respectively. Please zoom in for better results. As can be seen, 3DGS contains aliasing artifacts (e.g., 1st column). StableSR changes the contents of the rendered view (e.g., lego in the 3rd row). Analytic-Splatting and GaussianSR generate high-frequency artifacts. In contrast, the proposed method effectively renders high-fidelity results with rich details.