SA-GS: Scale-Adaptive Gaussian Splatting for Training-Free Anti-Aliasing

TL;DR

This work tackles aliasing and scale-mismatch in training-time dependent Gaussian Splatting by introducing SA-GS, a training-free, test-time framework that uses a 2D scale-adaptive filter to keep Gaussian projections consistent across rendering frequencies. The method enables straightforward anti-aliasing via super-sampling and integration, but only after scale-consistency is established, and it can be applied as a plugin to pretrained 3DGS models. Across Mip-NeRF 360 and Blender datasets, SA-GS achieves comparable or superior results to Mip-Splatting, while offering the advantage of not requiring retraining, and exhibits notable gains in zoom-out scenarios when using the full SA-GS pipeline. A small runtime overhead is noted for the anti-aliasing components, but the approach delivers meaningful quality improvements with minimal training overhead. The combination of a principled scale-adaptive filter and classical anti-aliasing techniques provides a practical, scalable solution for high-quality neural rendering with Gaussian primitives.

Abstract

In this paper, we present a Scale-adaptive method for Anti-aliasing Gaussian Splatting (SA-GS). While the state-of-the-art method Mip-Splatting needs modifying the training procedure of Gaussian splatting, our method functions at test-time and is training-free. Specifically, SA-GS can be applied to any pretrained Gaussian splatting field as a plugin to significantly improve the field's anti-alising performance. The core technique is to apply 2D scale-adaptive filters to each Gaussian during test time. As pointed out by Mip-Splatting, observing Gaussians at different frequencies leads to mismatches between the Gaussian scales during training and testing. Mip-Splatting resolves this issue using 3D smoothing and 2D Mip filters, which are unfortunately not aware of testing frequency. In this work, we show that a 2D scale-adaptive filter that is informed of testing frequency can effectively match the Gaussian scale, thus making the Gaussian primitive distribution remain consistent across different testing frequencies. When scale inconsistency is eliminated, sampling rates smaller than the scene frequency result in conventional jaggedness, and we propose to integrate the projected 2D Gaussian within each pixel during testing. This integration is actually a limiting case of super-sampling, which significantly improves anti-aliasing performance over vanilla Gaussian Splatting. Through extensive experiments using various settings and both bounded and unbounded scenes, we show SA-GS performs comparably with or better than Mip-Splatting. Note that super-sampling and integration are only effective when our scale-adaptive filtering is activated. Our codes, data and models are available at https://github.com/zsy1987/SA-GS.

Figures (16)

• Figure 1: Area scaling when rotating pixels. In integration method, the pixel area is scaled before projection to ensure that the projected (rotated) pixel area is equal to the original pixel area. $\theta$ is the rotation angle of the pixel.
• Figure 2: Paradigm Comparison of Gaussian Rasterization Process. All Gaussian Splatting methods share this framework for training and rendering, but different models use different strategies to process Gaussian primitives. During training, 3DGSkerbl20233dgs uses (c) in pixel space for training stability, but results in scale inconsistencies at different rendering settings; Mip-Splatting utilises (a) to restrict the Gaussian frequency upper bound in 3D space, and (b) to emulate box filtering in pixel space. But Mip-Splattingyu2023mip still suffers from scale inconsistency and needs to modify the training procedure of 3DGS. Our approach is training-free and only operates on the testing flow. We use (d) in pixel space to maintain the scale consistency of the Gaussian primitives, and further enhance the anti-aliasing capability of 3DGS by applying (e) and (f) to the $\alpha$-blending process. Note that (e) and (f) only make sense with (d) activated.
• Figure 2: Visual demonstration of our theoretical analysis. We scale the general Gaussian distribution to a standard normal distribution to estimate an upper bound on the error between the rotated pixel and the original pixel.
• Figure 3: Scale ambiguity. The heuristic 2D dilation process in vanilla 3DGS code (pointed out by yu2023mip) operates on the pixel space and enlarges the projected 2D Gaussian by a fixed amount (around 1.64 pixel). However, a fixed 2D dilation (1.64 pixel) can result in scale ambiguities when representing the same scene at different rendering settings, as shown by the green expansion area. (a) When the Gaussian scale is held constant and the resolution changes, the dilation scale (green) changes inconsistently. (b) When the Gaussian scale changes and the resolution remains constant, the dilation scale (green) does not change with the Gaussian. Our 2D scale-adaptive filter ensures that the Gaussian scale remains consistent across different rendering settings, as shown by the red expansion area. This keeps the scale consistent with the training setup.
• Figure 3: Numerical Experimental Results of Integration Error. we convert all the errors to the (0,1) range after transformation and widened the differences in the region close to 0. The average relative error is 0.51%, verifying that our method is a good estimation.