D2SL: Decouple Defogging and Semantic Learning for Foggy Domain-Adaptive Segmentation

TL;DR

The paper addresses the challenge of foggy domain-adaptive semantic segmentation by decoupling defogging from semantic learning. It introduces Domain-Consistent Transfer (DCT) to align defogging and segmentation features on clean data and Real Fog Transfer (RFT) to exploit real fog priors via Fog Domain Migration and Segmentation-Enhanced Defogging losses, followed by semantic fine-tuning that preserves defogging capability. Across synthetic and real fog datasets, D2SL achieves superior fog segmentation performance and demonstrates robust generalization to unseen clear-weather data. This approach enables more reliable perception in foggy conditions by effectively leveraging real fog data while maintaining semantic fidelity.

Abstract

We investigated domain adaptive semantic segmentation in foggy weather scenarios, which aims to enhance the utilization of unlabeled foggy data and improve the model's adaptability to foggy conditions. Current methods rely on clear images as references, jointly learning defogging and segmentation for foggy images. Despite making some progress, there are still two main drawbacks: (1) the coupling of segmentation and defogging feature representations, resulting in a decrease in semantic representation capability, and (2) the failure to leverage real fog priors in unlabeled foggy data, leading to insufficient model generalization ability. To address these issues, we propose a novel training framework, Decouple Defogging and Semantic learning, called D2SL, aiming to alleviate the adverse impact of defogging tasks on the final segmentation task. In this framework, we introduce a domain-consistent transfer strategy to establish a connection between defogging and segmentation tasks. Furthermore, we design a real fog transfer strategy to improve defogging effects by fully leveraging the fog priors from real foggy images. Our approach enhances the semantic representations required for segmentation during the defogging learning process and maximizes the representation capability of fog invariance by effectively utilizing real fog data. Comprehensive experiments validate the effectiveness of the proposed method.

Figures (9)

• Figure 1: Impact of joint learning and decoupled learning.
• Figure 2: An overview of $\rm D^2SL$. (a) The Domain-Consistent Transfer strategy aligns features extracted by the defogging encoder with those extracted by the segmentation encoder on the corresponding clean images, thereby disentangling the defogging and segmentation tasks. (b) The Real Fog Transfer strategy enhances semantic features of defogged images from both synthetic and real fog datasets, making them highly similar to their respective clean images. By leveraging Domain-Consistent Transfer and Real Fog Transfer strategies during the pre-training phase, $\rm D^2SL$ prevents defogging features from influencing semantic expression while incorporating real fog priors.
• Figure 3: The training strategy of FDM. We utilize FDM to get the fog priors inherent in real foggy images.
• Figure 4: The fine-tuning loss. The fine-tuning loss consists of three parts: Foggy Segmentation loss, Clean Segmentation loss, and Prediction Consistency loss.
• Figure 5: Qualitative results on the real foggy datasets. (a) Input images. (b) Joint training. (c) $\rm D^2SL$. (d) Groundtruth.