OurDB: Ouroboric Domain Bridging for Multi-Target Domain Adaptive Semantic Segmentation

TL;DR

An ouroboric domain bridging (OurDB) framework is proposed, offering an efficient solution to the MTDA problem using a single teacher architecture, dynamically cycles through multiple target domains, aligning each domain individually to restrain the biased alignment problem, and utilizes Fisher information to minimize the forgetting of knowledge from previous target domains.

Abstract

Multi-target domain adaptation (MTDA) for semantic segmentation poses a significant challenge, as it involves multiple target domains with varying distributions. The goal of MTDA is to minimize the domain discrepancies among a single source and multi-target domains, aiming to train a single model that excels across all target domains. Previous MTDA approaches typically employ multiple teacher architectures, where each teacher specializes in one target domain to simplify the task. However, these architectures hinder the student model from fully assimilating comprehensive knowledge from all target-specific teachers and escalate training costs with increasing target domains. In this paper, we propose an ouroboric domain bridging (OurDB) framework, offering an efficient solution to the MTDA problem using a single teacher architecture. This framework dynamically cycles through multiple target domains, aligning each domain individually to restrain the biased alignment problem, and utilizes Fisher information to minimize the forgetting of knowledge from previous target domains. We also propose a context-guided class-wise mixup (CGMix) that leverages contextual information tailored to diverse target contexts in MTDA. Experimental evaluations conducted on four urban driving datasets (i.e., GTA5, Cityscapes, IDD, and Mapillary) demonstrate the superiority of our method over existing state-of-the-art approaches.

Figures (5)

• Figure 1: Concept comparison between previous works and ours for MTDA. (a) Previous methods adopt a multiple teacher architecture to address the multiple target adaptation challenge. (b) In contrast, ours employs a single teacher architecture, leveraging cyclic adaptation across multiple domains akin to the ouroboros, a symbol of cyclic renewal and self-sustainment.
• Figure 2: Overview of the proposed OurDB. The framework is illustrated as an example where the number of target domains is $K=2$. ODS cyclically selects one domain to be aligned from multiple target domains (red area). OurDB then learns the bridges generated by CGMix to gradually reduce the domain discrepancies between the source and selected target domains (blue area). AF-EMA adaptively adjusts the EMA coefficients based on Fisher information to prevent the teacher model from forgetting the knowledge about the previous target domain (green area).
• Figure 3: Qualitative comparison between the CoaST and OurDB on the 7 classes benchmark for the GTA5 $\rightarrow$ Cityscapes + IDD combination.
• Figure 4: Performance comparison with and without AF-EMA on 19 classes benchmark for GTA5 $\rightarrow$ Cityscapes $+$ IDD $+$ Mapillary combination.
• Figure 5: Qualitative comparison between ClassMix and CGMix on GTA5, Cityscapes, IDD, and Mapillary datasets. GTA5 is treated as the source domain, and the others are regarded as the target domains. The first row presents the results of bridges between GTA5 and Cityscapes, and the second and third rows depict the results between GTA5 and IDD, and GTA5 and Mapillary, respectively. The classes inserted into the target images are marked with a red outline.