Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bayesian Monocular Depth Refinement via Neural Radiance Fields

Arun Muthukkumar

TL;DR

This work tackles the challenge of refining monocular depth estimates, which often miss fine geometric detail due to ill-posedness. It introduces MDENeRF, an iterative framework that trains NeRFs on synthetically perturbed viewpoints to produce depth means and per-pixel uncertainties, and fuses these with an initial monocular depth using Bayesian inference. Per-pixel uncertainty from NeRF rendering guides selective refinement, injecting high-frequency geometry while preserving global structure, and the process is repeated for 2–3 iterations. The approach yields sharper depth boundaries and lower global error on SUN RGB-D indoor scenes, with well-calibrated uncertainty that correlates with actual depth error, enabling robust depth refinement for downstream vision tasks.

Abstract

Monocular depth estimation has applications in many fields, such as autonomous navigation and extended reality, making it an essential computer vision task. However, current methods often produce smooth depth maps that lack the fine geometric detail needed for accurate scene understanding. We propose MDENeRF, an iterative framework that refines monocular depth estimates using depth information from Neural Radiance Fields (NeRFs). MDENeRF consists of three components: (1) an initial monocular estimate for global structure, (2) a NeRF trained on perturbed viewpoints, with per-pixel uncertainty, and (3) Bayesian fusion of the noisy monocular and NeRF depths. We derive NeRF uncertainty from the volume rendering process to iteratively inject high-frequency fine details. Meanwhile, our monocular prior maintains global structure. We demonstrate superior performance on key metrics and experiments using indoor scenes from the SUN RGB-D dataset.

Bayesian Monocular Depth Refinement via Neural Radiance Fields

TL;DR

This work tackles the challenge of refining monocular depth estimates, which often miss fine geometric detail due to ill-posedness. It introduces MDENeRF, an iterative framework that trains NeRFs on synthetically perturbed viewpoints to produce depth means and per-pixel uncertainties, and fuses these with an initial monocular depth using Bayesian inference. Per-pixel uncertainty from NeRF rendering guides selective refinement, injecting high-frequency geometry while preserving global structure, and the process is repeated for 2–3 iterations. The approach yields sharper depth boundaries and lower global error on SUN RGB-D indoor scenes, with well-calibrated uncertainty that correlates with actual depth error, enabling robust depth refinement for downstream vision tasks.

Abstract

Monocular depth estimation has applications in many fields, such as autonomous navigation and extended reality, making it an essential computer vision task. However, current methods often produce smooth depth maps that lack the fine geometric detail needed for accurate scene understanding. We propose MDENeRF, an iterative framework that refines monocular depth estimates using depth information from Neural Radiance Fields (NeRFs). MDENeRF consists of three components: (1) an initial monocular estimate for global structure, (2) a NeRF trained on perturbed viewpoints, with per-pixel uncertainty, and (3) Bayesian fusion of the noisy monocular and NeRF depths. We derive NeRF uncertainty from the volume rendering process to iteratively inject high-frequency fine details. Meanwhile, our monocular prior maintains global structure. We demonstrate superior performance on key metrics and experiments using indoor scenes from the SUN RGB-D dataset.
Paper Structure (17 sections, 16 equations, 3 figures, 2 tables)

This paper contains 17 sections, 16 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Monocular Depth Estimation
  4. Neural Rendering and Depth Estimation
  5. Depth Map Fusion and Refinement
  6. MDENeRF: Methodology
  7. Synthetic Data Generation
  8. Initial Depth Estimation
  9. NeRF Depth and Uncertainty
  10. Novel View Synthesis and Depth Reprojection
  11. Bayesian Depth Fusion
  12. Iterative Refinement Loop
  13. Experiments
  14. Quantitative Results
  15. Qualitative Results
  16. ...and 2 more sections

Figures (3)

  • Figure 1: One MDENeRF refinement iteration. A NeRF trains on nearby synthetic views and renders depth and uncertainty. NeRF and monocular depth estimates are fused via Bayesian inference. The process repeats for 2–3 iterations, which progressively enhances detail.
  • Figure 2: Correlation between predicted uncertainty and depth error.
  • Figure 3: Qualitative results on challenging SUN RGB-D scenes. MDENeRF sharpens thin structures (e.g., chair legs and lamp poles) and strengthens occlusion boundaries while preserving planar regions such as walls and floors.