AutoWeather4D: Autonomous Driving Video Weather Conversion via G-Buffer Dual-Pass Editing

Abstract

Generative video models have significantly advanced the photorealistic synthesis of adverse weather for autonomous driving; however, they consistently demand massive datasets to learn rare weather scenarios. While 3D-aware editing methods alleviate these data constraints by augmenting existing video footage, they are fundamentally bottlenecked by costly per-scene optimization and suffer from inherent geometric and illumination entanglement. In this work, we introduce AutoWeather4D, a feed-forward 3D-aware weather editing framework designed to explicitly decouple geometry and illumination. At the core of our approach is a G-buffer Dual-pass Editing mechanism. The Geometry Pass leverages explicit structural foundations to enable surface-anchored physical interactions, while the Light Pass analytically resolves light transport, accumulating the contributions of local illuminants into the global illumination to enable dynamic 3D local relighting. Extensive experiments demonstrate that AutoWeather4D achieves comparable photorealism and structural consistency to generative baselines while enabling fine-grained parametric physical control, serving as a practical data engine for autonomous driving.

Figures (28)

• Figure 1: AutoWeather4D: Weather & Time-of-Day Control for Driving Videos. AutoWeather4D enables fine-grained control over weather (rain, snow, fog) and time-of-day (dawn, noon, night) in driving videos. The red zoom-in box showcases realistic snow accumulation, while the blue zoom-in box highlights rain-induced road wetness and ripples. See supplementary videos for dynamic visualizations.
• Figure 2: Overview of our framework. The pipeline formulates physically-grounded video editing for multi-weather and time-of-day synthesis. We first extract explicit G-buffers from the input video: metric depth $\mathbf{D}$ via feed-forward 4D reconstruction, alongside intrinsic material properties (normal $\mathbf{N}$, metallic $\mathbf{M}$, albedo $\mathbf{A}$, roughness $\mathbf{R}$) via an inverse renderer. The scene modifications are analytically resolved through the G-Buffer Dual-Pass Editing: (1) The Geometry Pass physically modulates $\mathbf{A, N, R}$ to instantiate explicit weather mechanics (e.g., snow, rain, ground wetness); (2) The Light Pass executes parametric illumination control, independently synthesizing detected local light sources and global environmental lighting (e.g., dawn, noon, blue hours) to reflect atmospheric and temporal shifts. Finally, the deterministic rendered sequence is processed by the VidRefiner. This terminal refiner synthesizes real-world sensor nuances while preserving the classical shading cues and explicit scene dynamics resolved in the dual-pass stages.
• Figure 3: Qualitative Comparisons of AutoWeather4D on Waymo Weather/ Time-of-day Conversions: Validating Physically Plausible and Fine-Grained Control for Autonomous Driving.
• Figure 4: Qualitative Comparisons with domain-specific architectures: Validating spatially anchoring weather effects. DR: DiffusionRenderer DiffusionRenderer.
• Figure 5: Ablation of 4D reconstruction. (a) Integer-quantized depth priors DiffusionRenderer induce severe spatial discretization and aliasing during local relighting. (b) The deployed feed-forward 4D reconstruction establishes a continuous floating-point manifold, enforcing smooth, artifact-free illumination gradients.