WeatherGS: 3D Scene Reconstruction in Adverse Weather Conditions via Gaussian Splatting

TL;DR

WeatherGS tackles the problem of reconstructing clean 3D scenes from multi-view imagery captured under adverse weather. It introduces a dense-to-sparse preprocessing pipeline with an Atmospheric Effect Filter ($AEF$) and a Lens Effect Detector ($LED$), followed by training a $3DGS$ model with occlusion masks. The approach yields weather-free 3D reconstructions with fast rendering and provides a diverse benchmark for evaluating methods under snowy and rainy conditions. Experiments show WeatherGS consistently outperforms state-of-the-art 2D weather removal and NeRF/3DGS baselines in PSNR and perceptual metrics.

Abstract

3D Gaussian Splatting (3DGS) has gained significant attention for 3D scene reconstruction, but still suffers from complex outdoor environments, especially under adverse weather. This is because 3DGS treats the artifacts caused by adverse weather as part of the scene and will directly reconstruct them, largely reducing the clarity of the reconstructed scene. To address this challenge, we propose WeatherGS, a 3DGS-based framework for reconstructing clear scenes from multi-view images under different weather conditions. Specifically, we explicitly categorize the multi-weather artifacts into the dense particles and lens occlusions that have very different characters, in which the former are caused by snowflakes and raindrops in the air, and the latter are raised by the precipitation on the camera lens. In light of this, we propose a dense-to-sparse preprocess strategy, which sequentially removes the dense particles by an Atmospheric Effect Filter (AEF) and then extracts the relatively sparse occlusion masks with a Lens Effect Detector (LED). Finally, we train a set of 3D Gaussians by the processed images and generated masks for excluding occluded areas, and accurately recover the underlying clear scene by Gaussian splatting. We conduct a diverse and challenging benchmark to facilitate the evaluation of 3D reconstruction under complex weather scenarios. Extensive experiments on this benchmark demonstrate that our WeatherGS consistently produces high-quality, clean scenes across various weather scenarios, outperforming existing state-of-the-art methods. See project page:https://jumponthemoon.github.io/weather-gs.

Figures (6)

• Figure 1: Rendering examples under adverse weather conditions. NeRF introduces blur effects and lens occlusions, while 3DGS reconstructs scenes with dense weather particles and similar occlusions, significantly obscuring visibility. In contrast, the proposed WeatherGS effectively removes these artifacts, reconstructing clean 3D scenes and rendering artifact-free images.
• Figure 2: The overview of WeatherGS. The WeatherGS preprocesses multi-view images by removing weather-related visual artifacts. An atmospheric filter removes dense particles like raindrops and snowflakes, while a lens effect detector identifies and masks occlusions caused by precipitation on the camera lens. To ensure high-quality 3D scene reconstruction, we train the preprocessed images with 3D Gaussian splatting excluding the occlusion area to model the clear scene's geometry and appearance.
• Figure 3: The pipeline of atmospheric effect filter. A snowy scene is processed using the text prompt "Remove the snowy effect in the image." The image passes through encoder layers, and the system selects the "Desnow" task via similarity functions to remove snowflakes.
• Figure 4: Qualitative results on synthetic and real-world datasets under snowy and rainy conditions. NeRF-based methods show blurring and inaccurately rendered regions, while 3DGS-based approaches struggle to remove weather artifacts and camera occlusions. In contrast, WeatherGS successfully eliminates these weather-related disturbances, producing clearer, high-quality rendering results in both scenarios.
• Figure 5: The comparison results of with and without AEF and LED.