Solution for CVPR 2024 UG2+ Challenge Track on All Weather Semantic Segmentation

TL;DR

This work tackles semantic segmentation in adverse weather using the WeatherProof dataset to benchmark real-world degradations. The authors build an InternImage-H based segmentation pipeline with a UPerNet-style decoder, augmented with layer normalization and FFN, and trained with both offline and online data augmentation. They also employ model fusion via hard voting to refine predictions. On the WeatherProof benchmark, their approach yields strong robustness and places 3rd in CVPR 2024 UG2+ Challenge, demonstrating the value of large foundation-model backbones combined with task-specific augmentation for weather-robust segmentation.

Abstract

In this report, we present our solution for the semantic segmentation in adverse weather, in UG2+ Challenge at CVPR 2024. To achieve robust and accurate segmentation results across various weather conditions, we initialize the InternImage-H backbone with pre-trained weights from the large-scale joint dataset and enhance it with the state-of-the-art Upernet segmentation method. Specifically, we utilize offline and online data augmentation approaches to extend the train set, which helps us to further improve the performance of the segmenter. As a result, our proposed solution demonstrates advanced performance on the test set and achieves 3rd position in this challenge.

Figures (2)

• Figure 1: Overall Architecture of InternImage, where the core operator is DCNv3, and the basic block composes of layer normalization (LN) ba2016layer and feed-forward network (FFN) vaswani2017attention as transformers, the stem and downsampling layers follows conventional CNN’s designs
• Figure 2: Visualization of Select Segmentation Outputs