FourieRF: Few-Shot NeRFs via Progressive Fourier Frequency Control

TL;DR

FourieRF tackles the few-shot neural radiance field problem by introducing a progressive Fourier frequency curriculum that gradually increases scene detail. By applying a per-iteration frequency clipping to grid-based NeRF representations (CP or VM), it achieves fast convergence and robust geometry across synthetic and real scenes without relying on data priors. The method demonstrates state-of-the-art or competitive performance with substantially shorter training times, outperforming many data-prior and prior-free baselines in both speed and quality. This approach offers a practical, simple baseline for few-shot rendering and suggests avenues for augmenting it with large-data priors in the future.

Abstract

In this work, we introduce FourieRF, a novel approach for achieving fast and high-quality reconstruction in the few-shot setting. Our method effectively parameterizes features through an explicit curriculum training procedure, incrementally increasing scene complexity during optimization. Experimental results show that the prior induced by our approach is both robust and adaptable across a wide variety of scenes, establishing FourieRF as a strong and versatile baseline for the few-shot rendering problem. While our approach significantly reduces artifacts, it may still lead to reconstruction errors in severely under-constrained scenarios, particularly where view occlusion leaves parts of the shape uncovered. In the future, our method could be enhanced by integrating foundation models to complete missing parts using large data-driven priors.

Figures (7)

• Figure 1: FourieRF serves as an effective and simple baseline for tackling the few-shot rendering problem. The vanilla approach often encounters high-frequency artifacts early in the optimization process. We introduce an explicit curriculum training procedure that gradually incorporates higher frequencies to mitigate this. This method ensures a stable training trajectory, eliminating major artifacts and enhancing overall rendering quality.
• Figure 2: Method illustration. From left to right. Feature vectors and matrices are initialized in the spatial space. They are projected using the FFT. The Fourier coefficients are clipped using the masking procedure. Finally, the inverse FFT is applied to retrieve the smoothed features.
• Figure 3: Coarse Geometry Extraction. Our method is capable of extracting correct coarse geometry from as little as 3 views. This coarse geometry remains relatively stable regardless of the number of views we input.
• Figure 4: Overfitting on the few-shot rendering problem. "Catastrophic overfitting" is a common behavior for standard NeRF representations on the few-shot rendering problem. Degenerate geometry is learned, which might result in plausible views near train inputs but does not generalize to novel views.
• Figure 5: Comparison on Blender Dataset. In the Lego scene, trained with 4 views, we compare the performance of FreeNeRF, ZeroRF, and our method. ZeroRF renders a compact and clean reconstruction of the scene, however, at the cost of omitting some key details. FreeNeRF fails in this new setting due to its reliance on complex occlusion regularization. Despite employing a simple prior, FourieRF accurately captures both geometry and appearance, demonstrating a faithful reconstruction of the scene's details.