GaSLight: Gaussian Splats for Spatially-Varying Lighting in HDR

TL;DR

GaSLight introduces HDR Gaussian Splats ($3DGS$) as a spatially varying HDR lighting representation and pairs it with a diffusion-based LDR-to-HDR estimator to recover bright light intensities from regular images. The method first upsamples dynamic range using a diffusion model to generate plausible per-light HDR content, then fits a $3DGS$ scene to model near-field, high-frequency lighting that can be directly rendered. It demonstrates state-of-the-art performance on HDR estimation and effective applications in virtual object insertion and text-to-HDR lighting, supported by a new unsaturated HDR dataset and evaluation on existing benchmarks. The approach offers a practical, render-ready lighting representation that integrates with standard renderers while handling complex shadows and specularities that arise from spatially varying illumination.

Abstract

We present GaSLight, a method that generates spatially-varying lighting from regular images. Our method proposes using HDR Gaussian Splats as light source representation, marking the first time regular images can serve as light sources in a 3D renderer. Our two-stage process first enhances the dynamic range of images plausibly and accurately by leveraging the priors embedded in diffusion models. Next, we employ Gaussian Splats to model 3D lighting, achieving spatially variant lighting. Our approach yields state-of-the-art results on HDR estimations and their applications in illuminating virtual objects and scenes. To facilitate the benchmarking of images as light sources, we introduce a novel dataset of calibrated and unsaturated HDR to evaluate images as light sources. We assess our method using a combination of this novel dataset and an existing dataset from the literature. Project page: https://lvsn.github.io/gaslight/

Figures (7)

• Figure 1: Overview of our HDR estimation pipeline. (a) We train a diffusion model to predict two exposures (brighter or darker) given an input image. (b) At inference time, we recursively apply the same network to obtain an exposure stack, which is subsequently merged to HDR.
• Figure 2: Overview of our 3D reconstruction pipeline. From a collection of LDR input images, we first convert them to HDR by using our HDR light intensity estimation network (bottom, see \ref{['sec:light_intensity_estimation']}). A 3D Gaussian Splatting (3DGS) representation is subsequently trained on the resulting HDR images (top).
• Figure 3: Comparison of images from the (a) Si-HDR hanji2022comparison and (b) BtP-HDR bolduc2023beyond datasets. Image crops are shown on the top row, while the bottom row shows the corresponding plot of the luminance, showing (a) a saturated sample from the Si-HDR dataset; and (b) an unsaturated sample from BtP-HDR. The large flat regions of maximum intensity in (a) correspond to saturated pixel values. The BtP-HDR dataset does not contain such saturation.
• Figure 4: Qualitative evaluation on the filtered Si-HDR dataset hanji2022comparison. From an LDR image (left), we compare the luminance of each method's predicted HDR (even rows) and the rendering of a glossy sphere illuminated by the prediction (odd rows). Our approach more accurately estimates light intensity, producing renders similar to those rendered using the ground truth HDR image.
• Figure 5: HDR predictions (top row) are manually clamped to simulate inconsistent HDR predictions (middle row). The spherical harmonics RGB representation in the GS reconstruction (bottom row) smooths the values and improves multi-view consistency. Images are displayed in linear EV-3 for clarity.