A Curriculum-style Self-training Approach for Source-Free Semantic Segmentation

TL;DR

This work tackles semantic segmentation under source-free domain shift by introducing ATP, a curriculum-style framework that first aligns target features through entropy-aware, easy-to-hard learning and then refines the model via complementary self-training using negative and positive pseudo labels. It further reduces intra-domain discrepancies with information propagation and semantic contrastive learning, and extends to black-box source models through knowledge distillation. Empirical results across synthetic-to-real, adverse-condition, and cross-domain benchmarks show state-of-the-art performance among SFDA methods and competitive results versus data-dependent baselines, with robust ablations validating each component. The approach demonstrates practical utility and privacy-preserving properties, supported by code availability for reproducibility and extension to server-client scenarios.

Abstract

Source-free domain adaptation has developed rapidly in recent years, where the well-trained source model is adapted to the target domain instead of the source data, offering the potential for privacy concerns and intellectual property protection. However, a number of feature alignment techniques in prior domain adaptation methods are not feasible in this challenging problem setting. Thereby, we resort to probing inherent domain-invariant feature learning and propose a curriculum-style self-training approach for source-free domain adaptive semantic segmentation. In particular, we introduce a curriculum-style entropy minimization method to explore the implicit knowledge from the source model, which fits the trained source model to the target data using certain information from easy-to-hard predictions. We then train the segmentation network by the proposed complementary curriculum-style self-training, which utilizes the negative and positive pseudo labels following the curriculum-learning manner. Although negative pseudo-labels with high uncertainty cannot be identified with the correct labels, they can definitely indicate absent classes. Moreover, we employ an information propagation scheme to further reduce the intra-domain discrepancy within the target domain, which could act as a standard post-processing method for the domain adaptation field. Furthermore, we extend the proposed method to a more challenging black-box source model scenario where only the source model's predictions are available. Extensive experiments validate that our method yields state-of-the-art performance on source-free semantic segmentation tasks for both synthetic-to-real and adverse conditions datasets. The code and corresponding trained models are released at \url{https://github.com/yxiwang/ATP}.

Figures (6)

• Figure 1: Illustration of the proposed ATP framework. The step A achieves feature alignment via the proposed curriculum-style entropy minimization, which restrains the target prediction from easy-to-hard samples. The step T enhances target feature representation learning by the proposed curriculum-style complementary self-training technique, which leverages the negative pseudo labeling and positive pseudo labeling. Finally, step P achieves an information propagation performing as standard post-processing to reduce the intra-domain discrepancy.
• Figure 2: Complementary pseudo labels. Blue refers to positive pseudo labels, and red refers to negative pseudo labels.
• Figure 3: Illustration of the black-box source-model scenario.
• Figure 4: Visualization for predicted segmentation masks on the GTA5$\rightarrow$Cityscapes task.
• Figure 5: Robustness to the negative threshold $\lambda_{neg}$ for the negative pseudo assignment method.