MTGS: Multi-Traversal Gaussian Splatting

TL;DR

MTGS addresses the challenge of synthesizing photorealistic driving scenes from multi-traversal data by proposing a multi-traversal scene graph that separates shared static geometry, traversal-specific appearance, and transient dynamics. It combines Gaussian splatting with appearance residuals and LiDAR-guided exposure alignment to enable high-fidelity novel-view synthesis across traversals. The approach demonstrates state-of-the-art results on the nuPlan dataset, with substantial improvements in LPIPS and depth-geometry accuracy over single-traversal baselines and prior multi-traversal methods. This work advances autonomous driving simulation by delivering more realistic, view-consistent reconstructions that can generalize to unseen traversals and viewpoints.

Abstract

Multi-traversal data, commonly collected through daily commutes or by self-driving fleets, provides multiple viewpoints for scene reconstruction within a road block. This data offers significant potential for high-quality novel view synthesis, which is crucial for applications such as autonomous vehicle simulators. However, inherent challenges in multi-traversal data often result in suboptimal reconstruction quality, including variations in appearance and the presence of dynamic objects. To address these issues, we propose Multi-Traversal Gaussian Splatting (MTGS), a novel approach that reconstructs high-quality driving scenes from arbitrarily collected multi-traversal data by modeling a shared static geometry while separately handling dynamic elements and appearance variations. Our method employs a multi-traversal dynamic scene graph with a shared static node and traversal-specific dynamic nodes, complemented by color correction nodes with learnable spherical harmonics coefficient residuals. This approach enables high-fidelity novel view synthesis and provides flexibility to navigate any viewpoint. We conduct extensive experiments on a large-scale driving dataset, nuPlan, with multi-traversal data. Our results demonstrate that MTGS improves LPIPS by 23.5% and geometry accuracy by 46.3% compared to single-traversal baselines. The code and data would be available to the public.

Figures (8)

• Figure 1: Multi-Traversal Gaussian Splatting (MTGS) could reconstruct high-fidelity driving scenes from multi-traversal data. All images are rendered from a MTGS model of the same road block. (a) This approach preferably handles variations in lighting and shadows, rendering views conditioned on the traversal index ( Trv #). (b) The extrapolation quality of MTGS is showcased. It maintains high visual quality, even with lateral shifts of 8 meters (i.e., two lanes). For clarity, we mark a fixed reference point across traversals with a red pin.
• Figure 2: Overview. MTGS reconstructs a scene graph from multi-traversal sensor sequences. The scene graph consists of three types of nodes. (a) The rendering of a traversal subgraph starts with a shared static node, representing the time-invariant part of the scene. (b) This is followed by an appearance node that applies traversal-specific appearance effects, such as lighting and shadows. (c) Finally, transient nodes are placed in the background. (d) We align exposure using the overlapping LiDAR point cloud to ensure lighting consistency within the subgraph. (e) Photometric loss and multiple geometric losses are applied to bootstrap the reconstruction fidelity.
• Figure 3: Novel-view performance when trained with more traversals. More traversals do not guarantee improvement on existing methods, while our design unleashes their significance. *: affine-aligned PSNR.
• Figure 4: Visual comparison. Compared to OmniRE chen2025omnire and 3DGS kerbl20233dgs, MTGS produces images in higher fidelity, effectively handles appearance variations, and robustly extrapolates to novel views. Notably, our transient node accurately captures moving shadows (red box).
• Figure 5: Illustration of training and test traversals. We select traversals distributed across multiple lanes and choose the isolated traversal with minimum overlaps.