CONDA: Continual Unsupervised Domain Adaptation Learning in Visual Perception for Self-Driving Cars

TL;DR

This work addresses the challenge of continual unsupervised domain adaptation for semantic segmentation in autonomous driving, where models must adapt to new unlabeled domains without accessing prior training data. It introduces CONDA, combining an unsupervised target-domain loss with a distribution-shift regularizer, and models the prior prediction distribution $q(\mathbf{y})$ via a Bijective Network $G$ so that $\log p(\mathbf{y}) = \log \pi(\mathbf{z}) + \log\left|\frac{\partial G(\mathbf{y}, \theta_G)}{\partial \mathbf{y}}\right|$, with the prior $\pi$ chosen as Normal. The distribution-shift term is formulated as an upper bound of the KL divergence, $D_{MaL}(p(\mathbf{y}), q(\mathbf{y})) = - \mathbb{E}_{\mathbf{y} \sim p(\mathbf{y})} \log q(\mathbf{y})$, enabling learning without source data. A multi-scale invertible bijector models $q(\mathbf{y})$, facilitating the capture of global segmentation structure and enabling effective regularization during continual updates. Experimental results on GTA5→Cityscapes→IDD→Mapillary show state-of-the-art performance in continual UDDA, with consistent mIoU gains over baselines and notable per-class improvements, highlighting the practical potential of privacy-preserving continual adaptation for real-world self-driving perception systems.

Abstract

Although unsupervised domain adaptation methods have achieved remarkable performance in semantic scene segmentation in visual perception for self-driving cars, these approaches remain impractical in real-world use cases. In practice, the segmentation models may encounter new data that have not been seen yet. Also, the previous data training of segmentation models may be inaccessible due to privacy problems. Therefore, to address these problems, in this work, we propose a Continual Unsupervised Domain Adaptation (CONDA) approach that allows the model to continuously learn and adapt with respect to the presence of the new data. Moreover, our proposed approach is designed without the requirement of accessing previous training data. To avoid the catastrophic forgetting problem and maintain the performance of the segmentation models, we present a novel Bijective Maximum Likelihood loss to impose the constraint of predicted segmentation distribution shifts. The experimental results on the benchmark of continual unsupervised domain adaptation have shown the advanced performance of the proposed CONDA method.

Figures (3)

• Figure 1: Unsupervised domain Adaptation trains the model on both labeled source data and unlabeled target data simultaneously by a single training stage. Meanwhile, Continual Unsupervised Domain Adaptation first trains on the labeled source data. Then, it continuously adapts the model to the new target domains and at each training stage, the model only accesses data at that stage.
• Figure 2: The proposed CONDA Framework. The input images $\mathbf{x}$ are first forward to the segmentation network $F$. Then, the segmentation maps $\mathbf{y}$ are projected into the deep latent space by the bijective network $G$ to measure the distribution shift of target predictions compared to the original distribution of source predictions.
• Figure 3: Qualitative Results. We compare our results with the baseline method without baseline method (AdvEnt) vu2019advent.