TranSplat: Instant Cross-Scene Object Relighting in Gaussian Splatting via Spherical Harmonic Transfer

TL;DR

TranSplat introduces a fast, object-level relighting method for Gaussian Splatting by leveraging a spherical-harmonic radiance-transfer identity. It infers spatially varying source and target environment maps from GS representations and directly modulates per-Gaussian SH coefficients, avoiding explicit BRDF estimation or iterative inverse rendering. A per-Gaussian shading-map approach and a lightweight shadow-baking step enhance realism in cross-scene insertions. Experiments on synthetic and real scenes show competitive relighting accuracy with orders-of-magnitude faster runtimes than traditional inverse-rendering baselines, enabling near real-time cross-scene object transfers.

Abstract

We present TranSplat, a method for fast and accurate object relighting for the 3D Gaussian Splatting (GS) framework when transferring a 3D object from a source GS scene to a target GS scene. TranSplat is based on a theoretical radiance transfer identity for cross-scene relighting of objects with radially symmetric BRDFs that involves only taking simple products of spherical harmonic appearance coefficients of the object, source, and target environment maps without any explicit computation of scene quantities (e.g., the BRDFs themselves). TranSplat is the first method to demonstrate how this theoretical identity may be used to perform relighting within the GS framework, and furthermore, by automatically inferring unknown source and target environment maps directly from the source and target scene GS representations. We evaluated TranSplat on several synthetic and real-world scenes and objects, demonstrating comparable 3D object relighting performance to recent conventional inverse rendering-based GS methods with a fraction of their runtime. While TranSplat is theoretically best-suited for radially symmetric BRDFs, results demonstrate that TranSplat still offers perceptually realistic renderings on real scenes and opens a valuable, lightweight path forward to relighting with the GS framework.

Figures (15)

• Figure 1: Example demonstrations of TranSplat relighting a LEGO bulldozer from a source environment (top left) to target environments (bottom left). We assume that 3D Gaussian Splatting (GS) kerbl3Dgaussians representations may be built for both the source and target scenes from multiple views (not shown here), and that the object may be extracted (segmented) and placed into the target scene with either automatic or manual supervision. TranSplat then relights the object's Gaussians to reflect appropriate shading and shadow effects for the target environment. Crucially, TranSplat requires no ground truth information on the source or target radiance distributions (i.e., environment maps), no decomposition of object materials, and uses lightweight operations taking 10 seconds or less of processing time.
• Figure 2: Overview of TranSplat. Given source and target scenes, TranSplat first fits a Gaussian Surfels Dai2024GaussianSurfels model to the source scene ($S$) using input images and object masks, and a standard 3D Gaussian Splatting model to the target scene ($T$). From each trained GS representation, TranSplat renders cube maps centered at the object’s location to estimate spatially varying environment maps (${L}_S$, ${L}_T$). Using these environment maps and per-Gaussian normals, TranSplat performs Spherical Harmonic Transfer, an analytical transformation function applied on each Gaussian’s spherical harmonic (SH) appearance coefficients by the ratio of target to source lighting coefficients, modulated by per-Gaussian shading maps (${H}$). Finally, a lightweight shadow-baking step uses the dominant light lobes of ${L}_T$ to generate spatially consistent shadows, and the merged model is rendered to produce novel target-scene views.
• Figure 3: Capture an explicit environment map directly from 3D Gaussian fields. We render six 90° views around a specified position to obtain a cubemap, which is then converted into an equirectangular environment map. The position corresponds to the object's original location to sample the source lighting or the target location where the object will be inserted.
• Figure 4: Conceptual schematic of the shading map computation for a simple scene. (a) The shading map measures the contribution of each point in an environment map on each Gaussian surfel $j$ of the object (here, a bunny). The contribution is based on the angular alignment of the surfel's normal $\mathbf{n}^j$ with the point. In this example, Gaussian $j$ would not be strongly affected by the point along $\mathbf{u}$ with orientation $(\theta,\phi)$. (b) Without applying the shading map, all Gaussians receive input from all environment map points, resulting in a uniform appearance, whereas with shading, accurate colors and shadows are rendered.
• Figure 5: Relighting results of TranSplat across various environments. We use city and sunset environments as the source lighting conditions. In addition to three target environment with distinct color tones and directional lighting, we also include, in the left four columns, the results of swapping the city and sunset lighting conditions. TranSplat produces relighting results that closely match the GT shading and color tone.