FreeFix: Boosting 3D Gaussian Splatting via Fine-Tuning-Free Diffusion Models

TL;DR

FreeFix tackles the generalization–fidelity trade-off in extrapolated view rendering by enabling fine-tuning-free enhancement of 3D Gaussian Splatting with pretrained image diffusion models. It introduces an interleaved 2D-3D refinement loop and confidence-guided denoising based on Fisher information, enabling multi-view consistency without video diffusion backbones. By using multi-level certainty masks and an overall guidance strategy, FreeFix achieves state-of-the-art performance among fine-tuning-free methods and remains competitive with fine-tuned approaches across LLFF, Mip-NeRF 360, and Waymo. The approach preserves DM generalization, reduces artifact leakage into refined views, and highlights uncertainty-aware guidance as a more stable alternative to naive uncertainty-based methods, though it notes limitations in severe-artifact cases and convergence speed during 3D updates.

Abstract

Neural Radiance Fields and 3D Gaussian Splatting have advanced novel view synthesis, yet still rely on dense inputs and often degrade at extrapolated views. Recent approaches leverage generative models, such as diffusion models, to provide additional supervision, but face a trade-off between generalization and fidelity: fine-tuning diffusion models for artifact removal improves fidelity but risks overfitting, while fine-tuning-free methods preserve generalization but often yield lower fidelity. We introduce FreeFix, a fine-tuning-free approach that pushes the boundary of this trade-off by enhancing extrapolated rendering with pretrained image diffusion models. We present an interleaved 2D-3D refinement strategy, showing that image diffusion models can be leveraged for consistent refinement without relying on costly video diffusion models. Furthermore, we take a closer look at the guidance signal for 2D refinement and propose a per-pixel confidence mask to identify uncertain regions for targeted improvement. Experiments across multiple datasets show that FreeFix improves multi-frame consistency and achieves performance comparable to or surpassing fine-tuning-based methods, while retaining strong generalization ability.

Figures (12)

• Figure 1: Overview of FreeFix. We present FreeFix, a method designed to improve the rendering results of extrapolated views in 3D Gaussian Splatting, without requiring fine-tuning of diffusion models. Experiments on multiple datasets show that FreeFix provides performance that is comparable to, or even superior to, most advanced methods that require fine-tuning.
• Figure 2: Method. FreeFix improves the rendering quality of extrapolated views in 3DGS without fine-tuning DMs, as illustrated in the bottom left of the pipeline. We propose an interleaved strategy that combines 2D and 3D refinement to utilize image diffusion models for generating multi-frame consistent results, as shown at the top of the pipeline. In the 2D refinement stage, we also introduce confidence guidance and overall guidance to enhance the quality and consistency of the denoising results.
• Figure 3: Masks Comparison. We aim to generate masks for guidance during denoising to fix artifacts in rendered RGBs. (a) Rendered opacity maps do not account for the presence of artifacts. (b) Uncertainty Masks are aware of artifacts; however, due to their numerical instability, the volume rendering processing can be overwhelmed by low-opacity Gaussians with large uncertainties. (c) The certainty mask we propose is numerically stable and robust against various types of artifacts.
• Figure 4: Multi-Level Certainty Masks. FreeFix employs multiple $\gamma_c$ to obtain multi-level certainty masks as guidance. Each level of mask guides a different stage of denoising. A small $\gamma_c$ with high overall certainty is used for the early stages of denoising, while a large $\gamma_c$ which offers greater accuracy, is applied during the later stages of denoising.
• Figure 5: Qualitative Comparisons on LLFF mildenhall2019llff and Mip-NeRF 360 barron2021mip. FreeFix demonstrates state-of-the-art performance on these two datasets.