Towards Robust Online Domain Adaptive Semantic Segmentation under Adverse Weather Conditions

TL;DR

RODASS addresses the challenge of robust online domain adaptation for semantic segmentation under adverse weather by introducing a Dynamic Hyper-parameter Controller, a Dynamic Ambiguous Patch Mask, and a Dynamic Source Class Mix. The approach detects domain shifts in real time, masks highly disturbed regions to curb error propagation, and augments target data with a class-aware source buffer to accelerate adaptation while reducing noise and cost. Empirical results on OnDA benchmarks show state-of-the-art performance with competitive throughput (~40 FPS) and improved backward adaptation, demonstrating strong robustness to continuous domain shifts. This work offers a practical, efficient framework for deployment in safety-critical systems like autonomous driving under dynamic weather conditions.

Abstract

Online Domain Adaptation (OnDA) is designed to handle unforeseeable domain changes at minimal cost that occur during the deployment of the model, lacking clear boundaries between the domain, such as sudden weather events. However, existing OnDA methods that rely solely on the model itself to adapt to the current domain often misidentify ambiguous classes amidst continuous domain shifts and pass on this erroneous knowledge to the next domain. To tackle this, we propose \textbf{RODASS}, a \textbf{R}obust \textbf{O}nline \textbf{D}omain \textbf{A}daptive \textbf{S}emantic \textbf{S}egmentation framework, which dynamically detects domain shifts and adjusts hyper-parameters to minimize training costs and error propagation. Specifically, we introduce the \textbf{D}ynamic \textbf{A}mbiguous \textbf{P}atch \textbf{Mask} (\textbf{DAP Mask}) strategy, which dynamically selects highly disturbed regions and masks these regions, mitigating error accumulation in ambiguous classes and enhancing the model's robustness against external noise in dynamic natural environments. Additionally, we present the \textbf{D}ynamic \textbf{S}ource \textbf{C}lass \textbf{Mix} (\textbf{DSC Mix}), a domain-aware mix method that augments target domain scenes with class-level source buffers, reducing the high uncertainty and noisy labels, thereby accelerating adaptation and offering a more efficient solution for online domain adaptation. Our approach outperforms state-of-the-art methods on widely used OnDA benchmarks while maintaining approximately 40 frames per second (FPS).

Figures (5)

• Figure 1: Motivation of our proposed RODASS. Existing TTDA svdp still faces catastrophic forgetting and slow adaptation. SOTA OnDAcolomer2023hamlet has difficulty in distinguishing visually similar classes (road and wall), leading to error propagation. Our method allows the model to focus on the contextual information of the current scene, achieving robust performance and competitive speed.
• Figure 2: The overview of our proposed RODASS. RODASS aims to detect and adapt to domain changes in real-time from the DH Controller, while simultaneously mitigating the ambiguity by DAP Mask and reducing the uncertainty by DSC Mix that arises from external environmental factors and continuous distribution shifts.
• Figure 3: Continuous qualitative comparisons of the same frame between HAMLET and RODASS. We show the results on (a) Increasing Storm and (b) Increasing Fog.
• Figure 4: RODASS can maintain satisfactory segmentation quality over (a) Increasing Storm and (b) Increasing Fog, demonstrating its robustness in real-world scenarios.
• Figure 5: Additional analysis on (a) mask ratio, (b) mask strategy, and (c) replayed buffer. We report FPS (black line) and h-mIoU for forward (F), backward (B), and total (T) adaptation.