Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SiNeRF: Sinusoidal Neural Radiance Fields for Joint Pose Estimation and Scene Reconstruction

Yitong Xia, Hao Tang, Radu Timofte, Luc Van Gool

TL;DR

SiNeRF tackles the systematic sub-optimality of joint optimization in NeRFmm when reconstructing real-world scenes and estimating camera intrinsics/extrinsics without annotations. It combines sinusoidal activations in the radiance mapping network (SIREN-MLP) with Mixed Region Sampling to provide stronger, region-aware supervision during training. Across LLFF scenes, SiNeRF yields substantial gains in both image synthesis quality and pose accuracy compared with NeRFmm baselines, and ablations show the complementary value of SiNeRF and MRS. The work advances end-to-end NVS with minimal priors and offers open-source code for reproducibility.

Abstract

NeRFmm is the Neural Radiance Fields (NeRF) that deal with Joint Optimization tasks, i.e., reconstructing real-world scenes and registering camera parameters simultaneously. Despite NeRFmm producing precise scene synthesis and pose estimations, it still struggles to outperform the full-annotated baseline on challenging scenes. In this work, we identify that there exists a systematic sub-optimality in joint optimization and further identify multiple potential sources for it. To diminish the impacts of potential sources, we propose Sinusoidal Neural Radiance Fields (SiNeRF) that leverage sinusoidal activations for radiance mapping and a novel Mixed Region Sampling (MRS) for selecting ray batch efficiently. Quantitative and qualitative results show that compared to NeRFmm, SiNeRF achieves comprehensive significant improvements in image synthesis quality and pose estimation accuracy. Codes are available at https://github.com/yitongx/sinerf.

SiNeRF: Sinusoidal Neural Radiance Fields for Joint Pose Estimation and Scene Reconstruction

TL;DR

SiNeRF tackles the systematic sub-optimality of joint optimization in NeRFmm when reconstructing real-world scenes and estimating camera intrinsics/extrinsics without annotations. It combines sinusoidal activations in the radiance mapping network (SIREN-MLP) with Mixed Region Sampling to provide stronger, region-aware supervision during training. Across LLFF scenes, SiNeRF yields substantial gains in both image synthesis quality and pose accuracy compared with NeRFmm baselines, and ablations show the complementary value of SiNeRF and MRS. The work advances end-to-end NVS with minimal priors and offers open-source code for reproducibility.

Abstract

NeRFmm is the Neural Radiance Fields (NeRF) that deal with Joint Optimization tasks, i.e., reconstructing real-world scenes and registering camera parameters simultaneously. Despite NeRFmm producing precise scene synthesis and pose estimations, it still struggles to outperform the full-annotated baseline on challenging scenes. In this work, we identify that there exists a systematic sub-optimality in joint optimization and further identify multiple potential sources for it. To diminish the impacts of potential sources, we propose Sinusoidal Neural Radiance Fields (SiNeRF) that leverage sinusoidal activations for radiance mapping and a novel Mixed Region Sampling (MRS) for selecting ray batch efficiently. Quantitative and qualitative results show that compared to NeRFmm, SiNeRF achieves comprehensive significant improvements in image synthesis quality and pose estimation accuracy. Codes are available at https://github.com/yitongx/sinerf.
Paper Structure (17 sections, 6 equations, 3 figures, 3 tables)

This paper contains 17 sections, 6 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methods
  4. NeRFmm Preliminary
  5. Camera Parameters Formation
  6. Improve Joint Optimization with SiNeRF
  7. Potential Sub-Optimality of NeRFmm
  8. SiNeRF Architecture
  9. Partial Benefits For the Joint Optimization
  10. Mixed Region Sampling (MRS)
  11. Testing
  12. Experiments
  13. Settings
  14. Quantitative Evaluations
  15. Qualitative Results
  16. ...and 2 more sections

Figures (3)

  • Figure 1: General overview of SiNeRF. Our proposed Mixed Region Sampling contains both key point ray candidates (in orange circles) and random ray candidates (in green crosses). The reconstruction loss updates both SiNeRF and camera parameters. We empirically scale $\sigma$ by 25 to avoid faded synthesis.
  • Figure 2: Qualitative results of our method on the LLFF dataset. Comparisons on pose trajectories between SiNeRF and COLMAP are displayed in the bottom-left corner for each scene.
  • Figure 3: Qualitative results of ablation study on Trex scene.