Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AH-OCDA: Amplitude-based Curriculum Learning and Hopfield Segmentation Model for Open Compound Domain Adaptation

Jaehyun Choi, Junwon Ko, Dong-Jae Lee, Junmo Kim

TL;DR

This paper tackles Open Compound Domain Adaptation for semantic segmentation by introducing AH-OCDA, a two-component framework that combines amplitude-based curriculum learning with a Hopfield segmentation model. The curriculum uses FFT-based low-frequency amplitude information to rank and progressively align the model from near-source to far-source compound domains, while the Hopfield memory maps target-domain features to the source distribution to handle unseen open domains. The two components operate in a complementary, orthogonal manner and are trained with an adversarial objective that includes fake-source samples to mitigate forgetting. Evaluations on GTA5→C-Driving and SYNTHIA→C-Driving, plus extended open domains Cityscapes and KITTI, demonstrate state-of-the-art performance and robust generalization across diverse domain shifts, validating the approach for practical, continuously changing domain landscapes.

Abstract

Open compound domain adaptation (OCDA) is a practical domain adaptation problem that consists of a source domain, target compound domain, and unseen open domain. In this problem, the absence of domain labels and pixel-level segmentation labels for both compound and open domains poses challenges to the direct application of existing domain adaptation and generalization methods. To address this issue, we propose Amplitude-based curriculum learning and a Hopfield segmentation model for Open Compound Domain Adaptation (AH-OCDA). Our method comprises two complementary components: 1) amplitude-based curriculum learning and 2) Hopfield segmentation model. Without prior knowledge of target domains within the compound domains, amplitude-based curriculum learning gradually induces the semantic segmentation model to adapt from the near-source compound domain to the far-source compound domain by ranking unlabeled compound domain images through Fast Fourier Transform (FFT). Additionally, the Hopfield segmentation model maps segmentation feature distributions from arbitrary domains to the feature distributions of the source domain. AH-OCDA achieves state-of-the-art performance on two OCDA benchmarks and extended open domains, demonstrating its adaptability to continuously changing compound domains and unseen open domains.

AH-OCDA: Amplitude-based Curriculum Learning and Hopfield Segmentation Model for Open Compound Domain Adaptation

TL;DR

This paper tackles Open Compound Domain Adaptation for semantic segmentation by introducing AH-OCDA, a two-component framework that combines amplitude-based curriculum learning with a Hopfield segmentation model. The curriculum uses FFT-based low-frequency amplitude information to rank and progressively align the model from near-source to far-source compound domains, while the Hopfield memory maps target-domain features to the source distribution to handle unseen open domains. The two components operate in a complementary, orthogonal manner and are trained with an adversarial objective that includes fake-source samples to mitigate forgetting. Evaluations on GTA5→C-Driving and SYNTHIA→C-Driving, plus extended open domains Cityscapes and KITTI, demonstrate state-of-the-art performance and robust generalization across diverse domain shifts, validating the approach for practical, continuously changing domain landscapes.

Abstract

Open compound domain adaptation (OCDA) is a practical domain adaptation problem that consists of a source domain, target compound domain, and unseen open domain. In this problem, the absence of domain labels and pixel-level segmentation labels for both compound and open domains poses challenges to the direct application of existing domain adaptation and generalization methods. To address this issue, we propose Amplitude-based curriculum learning and a Hopfield segmentation model for Open Compound Domain Adaptation (AH-OCDA). Our method comprises two complementary components: 1) amplitude-based curriculum learning and 2) Hopfield segmentation model. Without prior knowledge of target domains within the compound domains, amplitude-based curriculum learning gradually induces the semantic segmentation model to adapt from the near-source compound domain to the far-source compound domain by ranking unlabeled compound domain images through Fast Fourier Transform (FFT). Additionally, the Hopfield segmentation model maps segmentation feature distributions from arbitrary domains to the feature distributions of the source domain. AH-OCDA achieves state-of-the-art performance on two OCDA benchmarks and extended open domains, demonstrating its adaptability to continuously changing compound domains and unseen open domains.

Paper Structure

This paper contains 31 sections, 15 equations, 9 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. Domain Adaptation and Generalization
  4. Open Compound Domain Adaptation
  5. Fourier Transform for Domain Adaptation
  6. Associative Memory
  7. Method
  8. Preliminaries
  9. Amplitude-based Curriculum Learning
  10. Hopfield Segmentation Model
  11. Loss Function
  12. Experiments
  13. Experimental settings
  14. Baselines
  15. Results
  16. ...and 16 more sections

Figures (9)

  • Figure 1: Overview of the proposed pipeline. For the amplitude-based curriculum learning, we extract the amplitudes with the Fast Fourier Transform and measure the distance between each target image and the source domain images. The Hopfield segmentation model is frozen when training.
  • Figure 2: UMAP results of GTA5 and C-Driving when $K$ = 3.
  • Figure 3: Sample images from each curriculum when $K$ = 3.
  • Figure 4: The high-level illustration of prototype-based model and Hopfield network-based model on OCDA setting.
  • Figure 5: Qualitative analysis on 'Cloudy' domain of GTA5 $\to$ C-Driving
  • ...and 4 more figures