IIDM: Inter and Intra-domain Mixing for Semi-supervised Domain Adaptation in Semantic Segmentation

TL;DR

The paper addresses domain shift in semantic segmentation by introducing IIDM, a semi-supervised domain adaptation framework that jointly exploits inter-domain mixing (source with unlabeled target) and intra-domain mixing (labeled target with unlabeled target). By using a mean-teacher setup with pseudo-labels, and losses for supervised source/target data plus inter- and intra-domain unsupervised losses, IIDM learns more domain-invariant features. Empirical results on GTA5→Cityscapes and SYNTHIA→Cityscapes show significant performance gains over prior SSDA methods, especially when labeled target data are scarce, and improvements persist when combined with advanced UDA techniques. The work highlights the importance of incorporating intra-domain target information and provides insights into effective domain-mixing strategies, with practical impact for real-world deployment where some labeled target data are available.

Abstract

Despite recent advances in semantic segmentation, an inevitable challenge is the performance degradation caused by the domain shift in real applications. Current dominant approach to solve this problem is unsupervised domain adaptation (UDA). However, the absence of labeled target data in UDA is overly restrictive and limits performance. To overcome this limitation, a more practical scenario called semi-supervised domain adaptation (SSDA) has been proposed. Existing SSDA methods are derived from the UDA paradigm and primarily focus on leveraging the unlabeled target data and source data. In this paper, we highlight the significance of exploiting the intra-domain information between the labeled target data and unlabeled target data. Instead of solely using the scarce labeled target data for supervision, we propose a novel SSDA framework that incorporates both Inter and Intra Domain Mixing (IIDM), where inter-domain mixing mitigates the source-target domain gap and intra-domain mixing enriches the available target domain information, and the network can capture more domain-invariant features. We also explore different domain mixing strategies to better exploit the target domain information. Comprehensive experiments conducted on the GTA5 to Cityscapes and SYNTHIA to Cityscapes benchmarks demonstrate the effectiveness of IIDM, surpassing previous methods by a large margin.

Figures (4)

• Figure 1: Comparison between state-of-the-art methods and the proposed Inter and Intra-Domain Mixing (IIDM) on the GTA5→Cityscapes setting. IIDM demonstrates superior performance across varying quantities of labeled target images, especially when the labeled target data is scarce.
• Figure 2: Motivation for using both inter-domain and intra-domain data mixing, where Gap1 means the domain gap between inter-domain mixed data center and the real target data center. Gap2 denotes the gap between intra-domain mixed data center and the real target data center. Learning with both inter and intra-domain data mixing equals to the learning based on the pseudo data center (Ours) which is the closest to the real target data center.
• Figure 3: Overview of the proposed IIDM. Note that the supervised branches on source data and labeled target data are omitted for brevity. Inter-domain mixing (green line) is applied between source data and unlabeled target data, while intra-domain mixing (orange line) is applied between labeled target data and unlabeled target data. Additionally, pseudo-labels are generated through an exponential moving average (EMA) teacher model (brown line), based on the unlabeled target data without mixing.
• Figure 4: Qualitative comparison of IIDM (row 2 and row 4) with previous methods (DLDM for row 1 and ComplexMix for row 3) on GTA5 to Cityscapes. IIDM better segments difficult classes such as sidewalk and truck.