Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DeRainGS: Gaussian Splatting for Enhanced Scene Reconstruction in Rainy Environments

Shuhong Liu, Xiang Chen, Hongming Chen, Quanfeng Xu, Mingrui Li

TL;DR

This study introduces the novel task of 3D Reconstruction in Rainy Environments (3DRRE), specifically designed to address the complexities of reconstructing 3D scenes under rainy conditions, and proposes DeRainGS, the first 3DGS method tailored for reconstruction in adverse rainy environments.

Abstract

Reconstruction under adverse rainy conditions poses significant challenges due to reduced visibility and the distortion of visual perception. These conditions can severely impair the quality of geometric maps, which is essential for applications ranging from autonomous planning to environmental monitoring. In response to these challenges, this study introduces the novel task of 3D Reconstruction in Rainy Environments (3DRRE), specifically designed to address the complexities of reconstructing 3D scenes under rainy conditions. To benchmark this task, we construct the HydroViews dataset that comprises a diverse collection of both synthesized and real-world scene images characterized by various intensities of rain streaks and raindrops. Furthermore, we propose DeRainGS, the first 3DGS method tailored for reconstruction in adverse rainy environments. Extensive experiments across a wide range of rain scenarios demonstrate that our method delivers state-of-the-art performance, remarkably outperforming existing occlusion-free methods.

DeRainGS: Gaussian Splatting for Enhanced Scene Reconstruction in Rainy Environments

TL;DR

This study introduces the novel task of 3D Reconstruction in Rainy Environments (3DRRE), specifically designed to address the complexities of reconstructing 3D scenes under rainy conditions, and proposes DeRainGS, the first 3DGS method tailored for reconstruction in adverse rainy environments.

Abstract

Reconstruction under adverse rainy conditions poses significant challenges due to reduced visibility and the distortion of visual perception. These conditions can severely impair the quality of geometric maps, which is essential for applications ranging from autonomous planning to environmental monitoring. In response to these challenges, this study introduces the novel task of 3D Reconstruction in Rainy Environments (3DRRE), specifically designed to address the complexities of reconstructing 3D scenes under rainy conditions. To benchmark this task, we construct the HydroViews dataset that comprises a diverse collection of both synthesized and real-world scene images characterized by various intensities of rain streaks and raindrops. Furthermore, we propose DeRainGS, the first 3DGS method tailored for reconstruction in adverse rainy environments. Extensive experiments across a wide range of rain scenarios demonstrate that our method delivers state-of-the-art performance, remarkably outperforming existing occlusion-free methods.
Paper Structure (24 sections, 13 equations, 7 figures, 3 tables)

This paper contains 24 sections, 13 equations, 7 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Image Deraining
  4. 3D Reconstruction in the Wild
  5. Dataset Construction
  6. Rain Streak Generation
  7. Raindrop Generation
  8. Real-world Data Collection
  9. Proposed Method
  10. Preliminaries: 3D Gaussian Splatting (3DGS)
  11. Rainy Image Enhancement
  12. Scene Reconstruction
  13. Frequency-based Feature Channel Attention
  14. Mask Prediction
  15. Optimization
  16. ...and 9 more sections

Figures (7)

  • Figure 1: DeRainGS reconstructs clean scenes from input rainy images. The visualization displays three different scenes along with the reconstruction outcomes of DeRainGS on our HydroViews dataset.
  • Figure 2: The visualization of our HydroViews dataset. From top to bottom, it includes synthesized raindrops, rain streaks, and real-world scenes.
  • Figure 3: The illustration of the DeRainGS pipeline. The left part displays the rainy image enhancement procedure and its network structure, which is pretrained before being applied to a scene. The right part demonstrates the reconstruction by 3DGS, utilizing learned occlusion masks to handle rain-induced artifacts.
  • Figure 4: Different types of artifacts, produced either by the image enhancement process or the persisting rain effect.
  • Figure 5: The visualization compares the rendering outcomes of DeRainGS with baseline methods on selected scenes featuring raindrops (top two rows) or rain streaks (bottom row) of our HydroViews dataset.
  • ...and 2 more figures