Technical Report for CVPR 2024 WeatherProof Dataset Challenge: Semantic Segmentation on Paired Real Data

TL;DR

The work tackles semantic segmentation under weather-induced image degradation using the WeatherProof dataset challenge, aiming for robust performance without external data. It leverages a large vision foundation model (InternImage-H) with Mask2Former as the detector baseline and investigates training/testing regimes, finding near-equal mIOU across strategies. To boost generalization, it introduces DA-Clip-based denoised augmentation (clean, noisy, denoised) used solely during training, and employs post-processing including model ensemble voting, dense-CRF, and morphological transformations. The approach achieves 45.1 mIOU and 2nd place, demonstrating the viability of combining foundation models with targeted augmentation and post-processing for weather-robust semantic segmentation in practical settings.

Abstract

This technical report presents the implementation details of 2nd winning for CVPR'24 UG2 WeatherProof Dataset Challenge. This challenge aims at semantic segmentation of images degraded by various degrees of weather from all around the world. We addressed this problem by introducing a pre-trained large-scale vision foundation model: InternImage, and trained it using images with different levels of noise. Besides, we did not use additional datasets in the training procedure and utilized dense-CRF as post-processing in the final testing procedure. As a result, we achieved 2nd place in the challenge with 45.1 mIOU and fewer submissions than the other winners.

Figures (1)

• Figure 1: Visualization of segmentation results, from top to bottom: raw image, baseline+model ensemble, baseline+model ensemble+dense-CRF, and baseline+model ensemble+dense-CRF+morphological transformation (final submitted).