Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mip-Grid: Anti-aliased Grid Representations for Neural Radiance Fields

Seungtae Nam, Daniel Rho, Jong Hwan Ko, Eunbyung Park

Abstract

Despite the remarkable achievements of neural radiance fields (NeRF) in representing 3D scenes and generating novel view images, the aliasing issue, rendering "jaggies" or "blurry" images at varying camera distances, remains unresolved in most existing approaches. The recently proposed mip-NeRF has addressed this challenge by rendering conical frustums instead of rays. However, it relies on MLP architecture to represent the radiance fields, missing out on the fast training speed offered by the latest grid-based methods. In this work, we present mip-Grid, a novel approach that integrates anti-aliasing techniques into grid-based representations for radiance fields, mitigating the aliasing artifacts while enjoying fast training time. The proposed method generates multi-scale grids by applying simple convolution operations over a shared grid representation and uses the scale-aware coordinate to retrieve features at different scales from the generated multi-scale grids. To test the effectiveness, we integrated the proposed method into the two recent representative grid-based methods, TensoRF and K-Planes. Experimental results demonstrate that mip-Grid greatly improves the rendering performance of both methods and even outperforms mip-NeRF on multi-scale datasets while achieving significantly faster training time. For code and demo videos, please see https://stnamjef.github.io/mipgrid.github.io/.

Mip-Grid: Anti-aliased Grid Representations for Neural Radiance Fields

Abstract

Despite the remarkable achievements of neural radiance fields (NeRF) in representing 3D scenes and generating novel view images, the aliasing issue, rendering "jaggies" or "blurry" images at varying camera distances, remains unresolved in most existing approaches. The recently proposed mip-NeRF has addressed this challenge by rendering conical frustums instead of rays. However, it relies on MLP architecture to represent the radiance fields, missing out on the fast training speed offered by the latest grid-based methods. In this work, we present mip-Grid, a novel approach that integrates anti-aliasing techniques into grid-based representations for radiance fields, mitigating the aliasing artifacts while enjoying fast training time. The proposed method generates multi-scale grids by applying simple convolution operations over a shared grid representation and uses the scale-aware coordinate to retrieve features at different scales from the generated multi-scale grids. To test the effectiveness, we integrated the proposed method into the two recent representative grid-based methods, TensoRF and K-Planes. Experimental results demonstrate that mip-Grid greatly improves the rendering performance of both methods and even outperforms mip-NeRF on multi-scale datasets while achieving significantly faster training time. For code and demo videos, please see https://stnamjef.github.io/mipgrid.github.io/.
Paper Structure (23 sections, 6 equations, 7 figures, 5 tables)

This paper contains 23 sections, 6 equations, 7 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Grid-based neural radiance fields
  4. Anti-aliasing and mipmap
  5. Method
  6. Background: grid representations for neural radiance fields
  7. Multi-resolution and multi-scale grid representations
  8. Shared grid and generating multi-scale grids
  9. Scale-aware input coordinates
  10. Mip-Grid
  11. Mip-TensoRF
  12. Mip-K-Planes
  13. Experiment
  14. Experimental setup
  15. Implementation details
  16. ...and 8 more sections

Figures (7)

  • Figure 1: (a) A quantitative comparison of mip-TensoRF against its three baseline models and mip-NeRF (see Sec. \ref{['sec:experiments']} for details). (b) Rendering samples of the lego scene in two different scales.
  • Figure 1: A qualitative comparison on each scene of the multi-scale Blender dataset.
  • Figure 2: Multi-resolution and multi-scale grid representations: the pt1 and pt2 are the same spatial coordinates in 3D space. When rendering from varying camera distances, (a) multi-resolution grid representations still retrieve the same features for both pt1 and pt2, while (b) multi-scale grid representations have distinct grids for each pt1 and pt2, hence, effectively resolving scale ambiguity.
  • Figure 2: A qualitative comparison on each scene of the multi-scale LLFF dataset.
  • Figure 3: The overall feature extraction pipeline in the proposed mip-TensoRF.
  • ...and 2 more figures