GLOW: Global Illumination-Aware Inverse Rendering of Indoor Scenes Captured with Dynamic Co-Located Light & Camera

TL;DR

GLOW tackles inverse rendering of multi-object indoor scenes under co-located light and camera by modeling global illumination with a dynamic radiance cache and a specularity-aware loss. It integrates neural implicit surface representation with a radiance cache to enable efficient, physically grounded rendering and joint optimization of geometry and BRDF. Key contributions include a geometry bootstrap via near-field lighting, a dynamic radiance cache that accommodates moving lights, and a surface-angle weighting loss that mitigates flashlight-induced artifacts, leading to substantial improvements in albedo and roughness while maintaining geometry quality. The approach enables effective relighting and material editing in realistic indoor scenes using accessible capture setups.

Abstract

Inverse rendering of indoor scenes remains challenging due to the ambiguity between reflectance and lighting, exacerbated by inter-reflections among multiple objects. While natural illumination-based methods struggle to resolve this ambiguity, co-located light-camera setups offer better disentanglement as lighting can be easily calibrated via Structure-from-Motion. However, such setups introduce additional complexities like strong inter-reflections, dynamic shadows, near-field lighting, and moving specular highlights, which existing approaches fail to handle. We present GLOW, a Global Illumination-aware Inverse Rendering framework designed to address these challenges. GLOW integrates a neural implicit surface representation with a neural radiance cache to approximate global illumination, jointly optimizing geometry and reflectance through carefully designed regularization and initialization. We then introduce a dynamic radiance cache that adapts to sharp lighting discontinuities from near-field motion, and a surface-angle-weighted radiometric loss to suppress specular artifacts common in flashlight captures. Experiments show that GLOW substantially outperforms prior methods in material reflectance estimation under both natural and co-located illumination.

Figures (18)

• Figure 1: We perform inverse rendering of a scene with non-trivial global illumination from multiple images captured with a co-located light and camera. The image on the left shows a rendering of the reconstructed scene under different illumination, next to the recovered geometry and material properties on the right.
• Figure 2: Qualitative comparison of reflectance estimation on synthetic scenes. We present estimated albedo, and roughness in validation views for the synthetic scenes. Top three rows are natural illumination methods while bottom five rows are co-located methods. Our method produces significantly better albedo and roughness w.r.t. natural illumination methods as co-located capture setup provides additional constraints. Prior co-located light & camera methods do not model global illumination and perform poorly. (Zoom in for better visualization.)
• Figure 3: Qualitative comparison of reflectance estimation on real scenes captured with co-located light & camera. We present estimated albedo, and roughness in validation views for the real scenes. Our method produces significantly better albedo and roughness w.r.t. prior co-located & camera methods due to our ability to better model global illumination. (Zoom for visualization.)
• Figure 4: Qualitative comparison of geometry with co-located light & camera methods. Our method produces better geometry than prior co-located light & camera methods.
• Figure 5: Qualitative results on ablation of radiance cache. For this ablation, we optimize material properties on ground truth mesh geometry of kitchen scene. Path is path tracing algorithm for reference. Direct is direct illumination renderer. Naive Cache is our algorithm but with naive radiance cache instead of dynamic radiance cache.