Animated 3DGS Avatars in Diverse Scenes with Consistent Lighting and Shadows

TL;DR

The paper addresses inconsistent lighting and missing shadows for animated 3D Gaussian Splatting avatars interacting with Gaussian scenes. It introduces Deep Gaussian Shadow Maps (DGSM), a volumetric shadowing approach that provides closed-form light accumulation along rays and stores transmittance over concentric shells in an octahedral atlas for real-time sampling, all within the Gaussian domain. Lighting compatibility is achieved via fast per-Gaussian radiance transfer from an approximate HDRI environment encoded in spherical harmonics (SH), enabling scene-consistent relighting without explicit BRDFs or offline optimization. The method is validated across single and multi-avatar setups and avatar–object interactions in diverse scenes, demonstrating coherent shadows and lighting while avoiding meshing. The approach offers a practical, efficient solution for integrating photorealistic avatars into 3DGS scenes with consistent illumination, paving the way for more seamless compositing and simulation workflows.

Abstract

We present a method for consistent lighting and shadows when animated 3D Gaussian Splatting (3DGS) avatars interact with 3DGS scenes or with dynamic objects inserted into otherwise static scenes. Our key contribution is Deep Gaussian Shadow Maps (DGSM), a modern analogue of the classical shadow mapping algorithm tailored to the volumetric 3DGS representation. Building on the classic deep shadow mapping idea, we show that 3DGS admits closed form light accumulation along light rays, enabling volumetric shadow computation without meshing. For each estimated light, we tabulate transmittance over concentric radial shells and store them in octahedral atlases, which modern GPUs can sample in real time per query to attenuate affected scene Gaussians and thus cast and receive shadows consistently. To relight moving avatars, we approximate the local environment illumination with HDRI probes represented in a spherical harmonic (SH) basis and apply a fast per Gaussian radiance transfer, avoiding explicit BRDF estimation or offline optimization. We demonstrate environment consistent lighting for avatars from AvatarX and ActorsHQ, composited into ScanNet++, DL3DV, and SuperSplat scenes, and show interactions with inserted objects. Across single and multi avatar settings, DGSM and SH relighting operate fully in the volumetric 3DGS representation, yielding coherent shadows and relighting while avoiding meshing.

Figures (4)

• Figure 2: Deep Gaussian Shadow Maps: For concentric spheres radiating out from light source, we build DGSMs by computing the light absorption by inserted Avatar/Object Gaussian at each radial distance from the light source. An octahedral map (right) takes a 3D unit vector $\textbf{d}$ and maps it to a 2D location in the atlas - of fixed dimension $H \times W$. Each of the absorption values in the concentric spheres is mapped to its own 2D octahedral map. The radial distances of the spheres are chunked into K discrete bins - along the radial direction $\textbf{t}$ - and stored in octahedral atlases. This creates a volumetric shadow map of fixed dimension $K \times H \times W$ which can be sampled to cast shadows on the Scene Gaussians.
• Figure 3: Qualitative Results: Our method casts plausible shadows for various Gaussian Avatars animated alone and with objects.
• Figure 4: Left: Without our relighting and shadow casting, the animated Avatar in 3D Gaussian scenes does not accurately reflect the lighting effects of the environment, nor does it cast shadows in the scene. The lighting of the Avatar remains uniform throughout. Right: With our SH based relighting and Deep Gaussian Shadow Map Formulation, the Avatar accurately reflects the lighting in the environment and casts accurate shadows.
• Figure 5: Ablation Studies: Variations of shadow map parameters. (a) Full method; (b) simple opacity-to-absorption; (c) diagonal opacity-to-absorption; (d) cubemaps instead of octahedral maps; (e) center sampling instead of MC; (f) deterministic samplings instead of MC.