ShinyNeRF: Digitizing Anisotropic Appearance in Neural Radiance Fields

TL;DR

ShinyNeRF addresses the challenge of accurately capturing anisotropic specular reflections in neural radiance fields for cultural heritage objects. It introduces an anisotropic reflectance model based on Anisotropic Spherical Gaussian (ASG) and approximates it with a symmetric mixture of von Mises-Fisher distributions, enabling joint estimation of normals, tangents, anisotropy e, and concentration kappa. The framework derives an Integrated Directional Encoding (IDE) in a reflection frame and uses a multi-term loss with geometry regularizers to ensure stable optimization and plausible material editing. Empirical results on synthetic datasets with varying anisotropic complexity demonstrate improved geometric fidelity and interpretable material parameters compared to baselines, highlighting ShinyNeRF’s potential for faithful digital preservation and editing of cultural artifacts.

Abstract

Recent advances in digitization technologies have transformed the preservation and dissemination of cultural heritage. In this vein, Neural Radiance Fields (NeRF) have emerged as a leading technology for 3D digitization, delivering representations with exceptional realism. However, existing methods struggle to accurately model anisotropic specular surfaces, typically observed, for example, on brushed metals. In this work, we introduce ShinyNeRF, a novel framework capable of handling both isotropic and anisotropic reflections. Our method is capable of jointly estimating surface normals, tangents, specular concentration, and anisotropy magnitudes of an Anisotropic Spherical Gaussian (ASG) distribution, by learning an approximation of the outgoing radiance as an encoded mixture of isotropic von Mises-Fisher (vMF) distributions. Experimental results show that ShinyNeRF not only achieves state-of-the-art performance on digitizing anisotropic specular reflections, but also offers plausible physical interpretations and editing of material properties compared to existing methods.

Figures (7)

• Figure 1: ShinyNeRF: novel view synthesis varying the learned reflectance parameters: anisotropy $e$ controls elongation of specular highlights, concentration $\kappa$ reduces the reflection sharpness and the tangent $\phi$ sets the anisotropy orientation.
• Figure 2: ShinyNeRF architecture. The spatial MLP is extended with additional parameters for the anisotropic reflectance model (anisotropy coefficient $e$, tangent orientation $\phi$). The outgoing specular radiance is encoded by a pretrained ASG2vMF network and then forwarded to the directional MLP responsible for rendering the specular color $\mathbf{c}_s$. Yellow blocks represent trainable MLPs, while blue blocks are non-learnable analytic expressions.
• Figure 3: ShinyNeRF anisotropic reflection model: the pretrained ASG2vMF network approximates the ASG ($\lambda$, $\mu$) in a canonical frame as a symmetric mixture of vMF functions, each defined by a concentration $\kappa_i$, an elevation $\theta_i$, and a weight $\alpha_i$ (not depicted). Vectors in red-green-blue denote orthonormal basis. The mean direction $\mathbf{z}_i$ of the vMFs is finally rotated according to the direction of reflection $\boldsymbol{\hat{\omega}}_r$.
• Figure 4: (a) ASG approximation error, (b) memory cost and (c) time per epoch for ShinyNeRF with varying number of vMF components $N$. The vertical dotted line marks the chosen number $N = 29$. Reported values use a training batch size of 3000 rays.
• Figure 5: ASG approximation with $N=29$ vMFs.