Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Physically-Based Sky-Modeling

Ian J. Maquignaz

TL;DR

This work tackles the challenge of photorealistic outdoor illumination by proposing AllSky, a deep neural network that learns weathered skies directly from physically captured HDR imagery. It introduces exposure-aware cascade losses and an ANN-based tonemapper to preserve Extended Dynamic Range ($EDR$) while enabling user control over sun and cloud configurations. The AllSky architecture uses a UNet backbone with dual HDR-expansion heads (ldr2EDR and latent2EDR) and priors, achieving improved dynamic-range retention and illumination coherence compared to baselines like DeepClouds and Text2Light, albeit with remaining issues in cloud texture fidelity and data limitations. Overall, the paper demonstrates a viable path toward replacing parametric sky-models in image-based lighting pipelines, while underscoring the need for richer HDRI datasets and more robust evaluation metrics.

Abstract

Accurate environment maps are a key component in rendering photorealistic outdoor scenes with coherent illumination. They enable captivating visual arts, immersive virtual reality and a wide range of engineering and scientific applications. Recent works have extended sky-models to be more comprehensive and inclusive of cloud formations but existing approaches fall short in faithfully recreating key-characteristics in physically captured HDRI. As we demonstrate, environment maps produced by sky-models do not relight scenes with the same tones, shadows, and illumination coherence as physically captured HDR imagery. Though the visual quality of DNN-generated LDR and HDR imagery has greatly progressed in recent years, we demonstrate this progress to be tangential to sky-modelling. Due to the Extended Dynamic Range (EDR) of 14EV required for outdoor environment maps inclusive of the sun, sky-modelling extends beyond the conventional paradigm of High Dynamic Range Imagery (HDRI). In this work, we propose an all-weather sky-model, learning weathered-skies directly from physically captured HDR imagery. Per user-controlled positioning of the sun and cloud formations, our model (AllSky) allows for emulation of physically captured environment maps with improved retention of the Extended Dynamic Range (EDR) of the sky.

Towards Physically-Based Sky-Modeling

TL;DR

This work tackles the challenge of photorealistic outdoor illumination by proposing AllSky, a deep neural network that learns weathered skies directly from physically captured HDR imagery. It introduces exposure-aware cascade losses and an ANN-based tonemapper to preserve Extended Dynamic Range () while enabling user control over sun and cloud configurations. The AllSky architecture uses a UNet backbone with dual HDR-expansion heads (ldr2EDR and latent2EDR) and priors, achieving improved dynamic-range retention and illumination coherence compared to baselines like DeepClouds and Text2Light, albeit with remaining issues in cloud texture fidelity and data limitations. Overall, the paper demonstrates a viable path toward replacing parametric sky-models in image-based lighting pipelines, while underscoring the need for richer HDRI datasets and more robust evaluation metrics.

Abstract

Accurate environment maps are a key component in rendering photorealistic outdoor scenes with coherent illumination. They enable captivating visual arts, immersive virtual reality and a wide range of engineering and scientific applications. Recent works have extended sky-models to be more comprehensive and inclusive of cloud formations but existing approaches fall short in faithfully recreating key-characteristics in physically captured HDRI. As we demonstrate, environment maps produced by sky-models do not relight scenes with the same tones, shadows, and illumination coherence as physically captured HDR imagery. Though the visual quality of DNN-generated LDR and HDR imagery has greatly progressed in recent years, we demonstrate this progress to be tangential to sky-modelling. Due to the Extended Dynamic Range (EDR) of 14EV required for outdoor environment maps inclusive of the sun, sky-modelling extends beyond the conventional paradigm of High Dynamic Range Imagery (HDRI). In this work, we propose an all-weather sky-model, learning weathered-skies directly from physically captured HDR imagery. Per user-controlled positioning of the sun and cloud formations, our model (AllSky) allows for emulation of physically captured environment maps with improved retention of the Extended Dynamic Range (EDR) of the sky.

Paper Structure

This paper contains 27 sections, 8 equations, 10 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Methodology
  4. Dynamic Range
  5. Losses
  6. Model
  7. Experiments
  8. Evaluation methodology
  9. Dataset
  10. Image-Processing
  11. Segmentation
  12. Baseline Models
  13. Baseline
  14. AllSky: ldr2EDR & latent2EDR
  15. DeepClouds
  16. ...and 12 more sections

Figures (10)

  • Figure 1: Impact of dynamic range on scene illumination. Each column illustrates the impact of an incremental clipping of dynamic range with exposure equalization to the 15EV ground truth.
  • Figure 2: Tone-Mapping Operators
  • Figure 3: Database sample, June 7th, 2016 at 1:54PM
  • Figure 4: AllSky
  • Figure 5: HDRDB Skydomes
  • ...and 5 more figures