Few TensoRF: Enhance the Few-shot on Tensorial Radiance Fields

Abstract

This paper presents Few TensoRF, a 3D reconstruction framework that combines TensorRF's efficient tensor based representation with FreeNeRF's frequency driven few shot regularization. Using TensorRF to significantly accelerate rendering speed and introducing frequency and occlusion masks, the method improves stability and reconstruction quality under sparse input views. Experiments on the Synthesis NeRF benchmark show that Few TensoRF method improves the average PSNR from 21.45 dB (TensorRF) to 23.70 dB, with the fine tuned version reaching 24.52 dB, while maintaining TensorRF's fast \(\approx10-15\) minute training time. Experiments on the THuman 2.0 dataset further demonstrate competitive performance in human body reconstruction, achieving 27.37 - 34.00 dB with only eight input images. These results highlight Few TensoRF as an efficient and data effective solution for real-time 3D reconstruction across diverse scenes.

Figures (8)

• Figure 1: Few TensoRF in the 3D NeRF-like Pipeline. The diagram shows where Few TensoRF fits into the core of the 3D NeRF-like reconstruction pipeline. It all begins with multiple scene images, and either COLMAP or Pose Diffusion calculates precise camera details. These ones, along with the images, train Few TensoRF. The trained Few TensoRF model then takes center stage in the final steps, ensuring a streamlined and accurate 3D model reconstruction within the NeRF-like framework. This visual guide underscores the seamless integration of Few TensoRF at the heart of the 3D NeRF-like process.
• Figure 2: The Few TensoRF overview. This illustration was adapted from the original TensoRF paper chen2022tensorf. It highlights two significant enhancements as we move from left to right: incorporating density frequency masks and integrating appearance color frequency on each rendered voxel. Additionally, a supplementary frequency mask is applied to refine the positional encoding fed into the neural networks.
• Figure 3: Comparing the rendered novel views quality of TensoRF baseline across different setups. To assess TensoRF’s performance on sparse input scenarios, we retrain the default TensoRF setup on different amounts of training image inputs. Following the image order, we have the original testing image (Ground Truth), and the novel view images rendered with TensoRF after training with 10 and 3 images (TensoRF_10img and TensorRF_10img) respectively.
• Figure 4: Diagram illustrating Frequency Masking on the appearance grid $G_c$ and view direction $d$.
• Figure 5: Comparing the view rendered of Hotdog object between methods.