Fitting Spherical Gaussians to Dynamic HDRI Sequences

TL;DR

This work tackles dynamic HDRI fitting by representing time-varying illumination with a temporally consistent mixture of anisotropic spherical Gaussians (ASGs). Each frame is modeled with a fixed number of ASGs and optimized using a composite loss that combines reconstruction accuracy, diffuse energy preservation, and temporal stability, with $L_R = \|I_{pred} - I_{gt}\|_1$, $L_D = \|D_{pred} - D_{gt}\|_1$, and $L_T = \sum_i \| (g_i^{t} - g_i^{t-1}) / \max_i\{ g_i^{t-1} \} \|_2$, while $I_{pred}(d) = \sum_i G_i(d)$ and $G_i(d) = c \exp(-\mu (d \cdot \mathbf{u}) - \lambda (d \cdot \mathbf{v}))$. The ASG parameters $g_i = (\mu_i, \lambda_i, \mathbf{u}_i, \mathbf{n}_i, c_i)$ are optimized per frame, with the first frame trained for 24{,}000 epochs and subsequent frames for 6{,}000 epochs using temporal regularization. The approach demonstrates that ~15 ASGs can effectively approximate ground-truth HDRIs across frequencies and roughness levels, enabling efficient rendering, but may struggle with fine reflections and sharp-angled lights, pointing to directions for future enhancement.

Abstract

We present a technique for fitting high dynamic range illumination (HDRI) sequences using anisotropic spherical Gaussians (ASGs) while preserving temporal consistency in the compressed HDRI maps. Our approach begins with an optimization network that iteratively minimizes a composite loss function, which includes both reconstruction and diffuse losses. This allows us to represent all-frequency signals with a small number of ASGs, optimizing their directions, sharpness, and intensity simultaneously for an individual HDRI. To extend this optimization into the temporal domain, we introduce a temporal consistency loss, ensuring a consistent approximation across the entire HDRI sequence.

Figures (4)

• Figure 1: Our method approximates an HDRI sequence using a temporally consistent mixture of anisotropic spherical Gaussians (ASGs). Below are three example frames showing our fitting results and the rendering of balls with varying roughness under the corresponding HDRI.
• Figure 2: The comparison of the ground truth HDRI and compression results with different numbers of ASGs, shown through latlongs and renderings.
• Figure 3: The comparison of different loss functions used for HDRI fitting with 15 ASGs. L1 refers to the L1 loss function used in the reconstruction loss; L2 indicates that the L1 loss is replaced by L2 loss; $L_D$ is the diffuse loss.
• Figure 4: The comparsion between with and without temporal consistency loss $L_T$ using 15 ASGs. The selected row from the HDRI sequence are stacked along the time axis to visualize temporal consistency.