Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BioNeRF: Biologically Plausible Neural Radiance Fields for View Synthesis

Leandro A. Passos, Douglas Rodrigues, Danilo Jodas, Kelton A. P. Costa, Ahsan Adeel, João Paulo Papa

TL;DR

Experimental results show that BioNeRF outperforms state-of-the-art results concerning a quality measure that encodes human perception in two datasets: real-world images and synthetic data.

Abstract

This paper presents BioNeRF, a biologically plausible architecture that models scenes in a 3D representation and synthesizes new views through radiance fields. Since NeRF relies on the network weights to store the scene's 3-dimensional representation, BioNeRF implements a cognitive-inspired mechanism that fuses inputs from multiple sources into a memory-like structure, improving the storing capacity and extracting more intrinsic and correlated information. BioNeRF also mimics a behavior observed in pyramidal cells concerning contextual information, in which the memory is provided as the context and combined with the inputs of two subsequent neural models, one responsible for producing the volumetric densities and the other the colors used to render the scene. Experimental results show that BioNeRF outperforms state-of-the-art results concerning a quality measure that encodes human perception in two datasets: real-world images and synthetic data.

BioNeRF: Biologically Plausible Neural Radiance Fields for View Synthesis

TL;DR

Experimental results show that BioNeRF outperforms state-of-the-art results concerning a quality measure that encodes human perception in two datasets: real-world images and synthetic data.

Abstract

This paper presents BioNeRF, a biologically plausible architecture that models scenes in a 3D representation and synthesizes new views through radiance fields. Since NeRF relies on the network weights to store the scene's 3-dimensional representation, BioNeRF implements a cognitive-inspired mechanism that fuses inputs from multiple sources into a memory-like structure, improving the storing capacity and extracting more intrinsic and correlated information. BioNeRF also mimics a behavior observed in pyramidal cells concerning contextual information, in which the memory is provided as the context and combined with the inputs of two subsequent neural models, one responsible for producing the volumetric densities and the other the colors used to render the scene. Experimental results show that BioNeRF outperforms state-of-the-art results concerning a quality measure that encodes human perception in two datasets: real-world images and synthetic data.
Paper Structure (14 sections, 10 equations, 3 figures, 4 tables)

This paper contains 14 sections, 10 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Biologically Plausible Neural Radiance Fields
  4. Methodology
  5. Datasets
  6. Blender
  7. Local Light Field Fusion
  8. Experimental Setup
  9. Experiments
  10. General Evaluation
  11. Quantitative evaluation
  12. Qualitative evaluation.
  13. Ablation study
  14. Discussions, Conclusions, and Future Works

Figures (3)

  • Figure 1: Biologically Plausible Neural Radiance Fields. The top-left frame describes the symbols, while the top-right frames depict the input/output variables. The bottom block illustrates the model's overall pipeline, comprising four steps. Step 1 describes the Positional Feature Extraction, which shall consist of two MLP blocks, namely $M_\Delta$ and $M_c$, responsible for extracting relevant information from the camera input to generate $\bm{h}_\Delta$ and $\bm{h}_c$. Step 2, i.e., Cognitive Filtering, illustrates the filters' generation process, while the Memory Updating in step 3 depicts the memory updating schema. Finally, Step 4, i.e., Contextual Inference, shows how the memory is filtered and concatenated with the camera position to feed $M^\prime_\Delta$ to generate $\Delta$ and combined with the camera angle to feed $M^\prime_c$ to generate $\bm{c}$. Notice that $\Delta$ and $\bm{c}$ are used to render the output compared against the pixel color to compute the BioNeRF's loss.
  • Figure 2: Qualitative results of BioNeRF and comparison methods, namely NeRF mildenhall2021nerf, Mip-NeRF 360 barron2022mip, and TensoRF chen2022tensorf, as well as the ground truth images (GT) on two Blender dataset's scenes.
  • Figure 3: Qualitative results of BioNeRF and comparison methods, namely, Mip-NeRF 360 barron2022mip and TensoRF chen2022tensorf, as well as the ground truth images (GT) on LLFF dataset's Fern scenes.