Mip-Splatting: Alias-free 3D Gaussian Splatting

TL;DR

The paper tackles aliasing and dilation artifacts in 3D Gaussian Splatting when rendering across varying sampling rates. It introduces a Nyquist-based 3D smoothing filter to cap the maximal frequency of 3D Gaussians and replaces screen-space dilation with a 2D Mip filter to emulate the physical imaging process, yielding alias-free renderings across scales. Experimental results on Blender and Mip-NeRF 360 demonstrate strong cross-scale generalization and competitive in-distribution performance, outperforming prior 3DGS variants, especially under zoomed-out or zoomed-in conditions. Overall, Mip-Splatting provides principled, lightweight modifications that improve robustness to scale while preserving rendering efficiency.

Abstract

Recently, 3D Gaussian Splatting has demonstrated impressive novel view synthesis results, reaching high fidelity and efficiency. However, strong artifacts can be observed when changing the sampling rate, \eg, by changing focal length or camera distance. We find that the source for this phenomenon can be attributed to the lack of 3D frequency constraints and the usage of a 2D dilation filter. To address this problem, we introduce a 3D smoothing filter which constrains the size of the 3D Gaussian primitives based on the maximal sampling frequency induced by the input views, eliminating high-frequency artifacts when zooming in. Moreover, replacing 2D dilation with a 2D Mip filter, which simulates a 2D box filter, effectively mitigates aliasing and dilation issues. Our evaluation, including scenarios such a training on single-scale images and testing on multiple scales, validates the effectiveness of our approach.

Figures (9)

• Figure 1: 3D Gaussian Splattingkerbl3Dgaussians renders images by representing 3D Objects as 3D Gaussians which are projected onto the image plane followed by 2D Dilation in screen space as shown in (a). The method's intrinsic shrinkage bias leads to degenerate 3D Gaussians exceed sampling limit as illustrated by the $\delta$ function in (b) while rendering similarly to 2D due to the dilation operation. However, when changing the sampling rate (via the focal length or camera distance), we observe strong dilation effects (c) and high frequency artifacts (d).
• Figure 2: We trained all the models on single-scale (full resolution here) images and rendered images with different resolutions by changing focal length. While all methods show similar performance at training scale, we observe strong artifacts in previous work kerbl3Dgaussianszwicker2001ewa when changing the sampling rate. By contrast, our Mip-Splatting renders faithful images across different scales.
• Figure 3: Sampling limits. A pixel corresponds to sampling interval $\hat{T}$. We band-limit the 3D Gaussians by the maximal sampling rate (i.e., minimal sampling interval) among all observations. This example shows 5 cameras at different depths $d$ and with different focal lengths $f$. Here, camera 3 determines the minimal $\hat{T}$ and hence the maximal sampling rate $\hat{\nu}$.
• Figure 4: Single-scale Training and Multi-scale Testing on the Blender Dataset mildenhall2020nerf. All methods are trained at full resolution and evaluated at different (smaller) resolutions to mimic zoom-out. Methods based on 3DGS capture fine details better than Mip-NeRF Barron2021ICCV and Tri-MipRF Hu2023ICCV at training resolution. Mip-Splatting surpasses both 3DGS kerbl3Dgaussians and 3DGS + EWA zwicker2001ewa at lower resolutions.
• Figure 5: Single-scale Training and Multi-scale Testing on the Mip-NeRF 360 Dataset barron2022mipnerf360. All models are trained on images downsampled by a factor of eight and rendered at full resolution to demonstrate zoom-in/moving closer effects. In contrast to prior work, Mip-Splatting renders images that closely approximate ground truth. Please also note the high-frequency artifacts of 3DGS + EWA zwicker2001ewa.