BRDF-NeRF: Neural Radiance Fields with Optical Satellite Images and BRDF Modelling

TL;DR

This work extends Neural Radiance Fields to remote sensing by embedding the Rahman-Pinty-Verstraete (RPV) BRDF into a NeRF framework, enabling realistic reflectance modeling of natural Earth surfaces from sparse satellite views. Depth supervision and guided sampling constrain geometry when only 3–4 views are available, while predicting BRDF parameters $\bm{\rho_0}$, $k$, $\bm{\Theta}$, and $\bm{\rho_c}$ and surface normals. The approach achieves superior novel-view synthesis and digital surface model quality on Djibouti and Lanzhou datasets compared to baselines like Sat-NeRF and SpS-NeRF, especially after atmospheric correction. Limitations include the absence of explicit shading modeling and full DSM regularization, suggesting avenues for future work in shading integration and DSM refinement.

Abstract

Neural radiance fields (NeRF) have gained prominence as a machine learning technique for representing 3D scenes and estimating the bidirectional reflectance distribution function (BRDF) from multiple images. However, most existing research has focused on close-range imagery, typically modeling scene surfaces with simplified Microfacet BRDF models, which are often inadequate for representing complex Earth surfaces. Furthermore, NeRF approaches generally require large sets of simultaneously captured images for high-quality surface depth reconstruction - a condition rarely met in satellite imaging. To overcome these challenges, we introduce BRDF-NeRF, which incorporates the physically-based semi-empirical Rahman-Pinty-Verstraete (RPV) BRDF model, known to better capture the reflectance properties of natural surfaces. Additionally, we propose guided volumetric sampling and depth supervision to enable radiance field modeling with a minimal number of views. Our method is evaluated on two satellite datasets: (1) Djibouti, captured at varying viewing angles within a single epoch with a fixed Sun position, and (2) Lanzhou, captured across multiple epochs with different Sun positions and viewing angles. Using only three to four satellite images for training, BRDF-NeRF successfully synthesizes novel views from unseen angles and generates high-quality digital surface models (DSMs).

Figures (16)

• Figure 1: BRDF-NeRF Workflow. A few satellite RGB images, and the corresponding low-resolution depth maps calculated using a classical stereo matching algorithm are fed into BRDF-NeRF to predict the normals n, and the RPV parameters $\bm{\rho_0}$, $\textbf{k}$, $\bm{\Theta}$ and $\bm{\rho_c}$, describing the surface reflectance. $\bm{\rho_0}$ represents the amplitude component, $\textbf{k}$ controls the overall shape of the anisotropic behaviour, $\bm{\Theta}$ establishes the degree of forward or backward scattering, and $\bm{\rho_c}$ allows to model the hotspot effect. The five parameters are integrated into a RPV renderer to generate the synthetic image. In the meantime, high resolution depths are obtained by accumulating the weights in the volume estimated by BRDF-NeRF.
• Figure 2: Scattering Patterns. Lambertian (a), Microfacet (d) and RPV (b, c, e, f) BRDF models. RPV can be used to simulate complex scattering indices.
• Figure 3: BRDF Defining Directions. Decomposition of the illumination $\textbf{w}_{ir}$ and viewing $\textbf{w}_{r}$ directions into ${\theta}_{ir}$ (solar zenith angle) and ${\theta}_{r}$ (viewing zenith angle), $\Phi_{ir}$ (solar azimuth angle) and $\Phi_{r}$ (viewing azimuth angle), g (phase angle) and $\Phi$ (relative azimuth angle).
• Figure 4: RPV Model Parameters -- BRF. Reflectances displayed in polar coordinates correspond to point A shown in (a). They are generated by evaluating the RPV model over a range of viewing directions, where $\bm{\rho_0}$ and $\textbf{n}$ are predicted by our BRDF-NeRF and fixed, while the Sun position is fixed and displayed as a white symbol. (b) backward scattering corresponding to the RPV parameters ($\bm{\Theta},\mathbf{k},\bm{\rho_c}$) found by BRDF-NeRF; (c) forward scattering obtained by modifying $\bm{\Theta}$; (d-e) transition between bowl shaped and bell shaped scattering obtained by changing the $\mathbf{k}$ parameter; (f) hotspot effect with modified $\bm{\rho_c}$. The combinations of the six parameters are provided in \ref{['BRF_circle_table']}.
• Figure 5: BRDF-NeRF Architecture. The input 3D locations $\textbf{x}$ are sampled along the camera rays and fed into BRDF-NeRF to query density and RPV parameters. BRDF-NeRF consists of shared 8-layer spatial MLPs, 1-layer feature MLP and four 1-layer RPV MLPs. The 8-layer MLPs are followed by a softplus function to predict the density $\sigma$, which is further used to calculate the analytical normal $\textbf{n}$. The RPV MLPs are concatenated with a sigmoid function to predict $\bm{\rho_0}$, $\textbf{k}$, $\bm{\Theta}$ and $\bm{\rho_c}$. The elements within the dashed rectangle are pre-trained on the assumption of a Lambertian surface for the first 20% of the total training steps. The colour $\textbf{c}$ is predicted with the RPV rendering equation in \ref{['renderingequation']}, where $W_{ir}$ and $W_r$ represent the Sun and camera directions, respectively.