EAG-PT: Emission-Aware Gaussians and Path Tracing for Indoor Scene Reconstruction and Editing

TL;DR

Indoor scene editing with radiance-field reconstructions often yields inconsistent illumination because light transport is not modeled. EAG-PT introduces Emission-Aware Gaussians and Path Tracing, a mesh-free pipeline that separates emitters from non-emissive geometry, recovers radiance and material with differentiable rendering, and uses multi-bounce path tracing for edited scenes while light-baking GI into Gaussians for efficiency. Experiments on synthetic and real indoor data show that EAG-PT produces physically consistent renders after edits, preserves fine geometry, and outperforms mesh-based inverse path tracing in quality and storage. This work enables practical interior design, XR content creation, and embodied AI asset preparation by providing a controllable, physically grounded representation and rendering workflow for edited indoor scenes.

Abstract

Recent reconstruction methods based on radiance field such as NeRF and 3DGS reproduce indoor scenes with high visual fidelity, but break down under scene editing due to baked illumination and the lack of explicit light transport. In contrast, physically based inverse rendering relies on mesh representations and path tracing, which enforce correct light transport but place strong requirements on geometric fidelity, becoming a practical bottleneck for real indoor scenes. In this work, we propose Emission-Aware Gaussians and Path Tracing (EAG-PT), aiming for physically based light transport with a unified 2D Gaussian representation. Our design is based on three cores: (1) using 2D Gaussians as a unified scene representation and transport-friendly geometry proxy that avoids reconstructed mesh, (2) explicitly separating emissive and non-emissive components during reconstruction for further scene editing, and (3) decoupling reconstruction from final rendering by using efficient single-bounce optimization and high-quality multi-bounce path tracing after scene editing. Experiments on synthetic and real indoor scenes show that EAG-PT produces more natural and physically consistent renders after editing than radiant scene reconstructions, while preserving finer geometric detail and avoiding mesh-induced artifacts compared to mesh-based inverse path tracing. These results suggest promising directions for future use in interior design, XR content creation, and embodied AI.

Figures (12)

• Figure 1: Scene editing on 2D Gaussian primitives of a reconstructed real indoor scene, f-classroom, including: a) relighting the ceiling, b) inserting colorful emissive balls, c) duplicating a chair with modified material, d) adding a diffuse ball, and e) importing a lamp from another scene. Path-traced rendering after editing produces coherent global illumination (reflections, interreflections, and shadows) in contrast to direct radiant scene composition.
• Figure 2: Renders of f-classroom before and after editing. Radiant Scene: Most radiance field reconstruction works NeRF3DGS3DGRT regard the whole scene as radiant, which cannot produce light changes and shadow effects after scene editing. Radiant Reflection: Some reflection modeling works I2SDFTexIR add a single bounce to produce more realistic results, while still suffering from the incorrect radiance after scene editing. Radiant Emission: We explicitly separate light sources from the radiant scene, and use path tracing to bounce light in the scene to derive photo-realistic renders.
• Figure 3: Pipeline of Emission-Aware Gaussians and Path Tracing. Given multi-view linear captures of an indoor scene with corresponding emitter masks and estimated normals, the radiant scene is first reconstructed in Stage 0 to get radiance, separate emitters, and derive geometry, based on 2D Gaussians and ray tracing. The material of the non-emitters is then recovered in Stage 1 through light bouncing and differentiable rendering. With properties of emitters, non-emitters, and scene geometry, path tracing that bounces light around the scene is adopted for photo-realistic renders on various scene editing scenarios.
• Figure 4: Relighting results with an inserted illuminated ball on synthetic scenes. Insets show FLIP error maps w.r.t. the relighting ground truth. While 0-bounce and 1-bounce renderings fail to reproduce global illumination after editing, our path tracing reproduces the target global illumination.
• Figure 5: Relighting results on the captured real scene lectureroom. For each relighting condition that turns off half lights, our path tracing closely reproduces the spatially varying indoor illumination compared with ground-truth relit photograph.