DomCLP: Domain-wise Contrastive Learning with Prototype Mixup for Unsupervised Domain Generalization

TL;DR

This work tackles the challenge of unsupervised domain generalization (UDG) in self-supervised learning by identifying that standard InfoNCE-based instance discrimination tends to suppress domain-irrelevant features and favor domain-relevant ones. It introduces DomCLP, which combines Domain-wise Contrastive Learning (DCon) to preserve domain-irrelevant features and Prototype Mixup Learning (PMix) to generalize across domains through multi-cluster prototypical mixups, augmented by prototypical contrastive learning. The method yields state-of-the-art results on PACS and DomainNet across label fractions, with notable gains at low supervision (e.g., 1%), and ablations confirm the value of each component and the importance of diverse clustering. These results suggest that domain-irrelevant feature learning and diverse prototype-based generalization can robustly handle unseen-domain shifts, with practical implications for real-world domain transfer tasks. All mathematical expressions are denoted with $…$ to maintain precise representation of the objective terms and losses.

Abstract

Self-supervised learning (SSL) methods based on the instance discrimination tasks with InfoNCE have achieved remarkable success. Despite their success, SSL models often struggle to generate effective representations for unseen-domain data. To address this issue, research on unsupervised domain generalization (UDG), which aims to develop SSL models that can generate domain-irrelevant features, has been conducted. Most UDG approaches utilize contrastive learning with InfoNCE to generate representations, and perform feature alignment based on strong assumptions to generalize domain-irrelevant common features from multi-source domains. However, existing methods that rely on instance discrimination tasks are not effective at extracting domain-irrelevant common features. This leads to the suppression of domain-irrelevant common features and the amplification of domain-relevant features, thereby hindering domain generalization. Furthermore, strong assumptions underlying feature alignment can lead to biased feature learning, reducing the diversity of common features. In this paper, we propose a novel approach, DomCLP, Domain-wise Contrastive Learning with Prototype Mixup. We explore how InfoNCE suppresses domain-irrelevant common features and amplifies domain-relevant features. Based on this analysis, we propose Domain-wise Contrastive Learning (DCon) to enhance domain-irrelevant common features. We also propose Prototype Mixup Learning (PMix) to generalize domain-irrelevant common features across multiple domains without relying on strong assumptions. The proposed method consistently outperforms state-of-the-art methods on the PACS and DomainNet datasets across various label fractions, showing significant improvements. Our code will be released. Our project page is available at https://github.com/jinsuby/DomCLP.

Figures (7)

• Figure 1: (a) In the UDG environment, representations include both domain-irrelevant common features and domain-relevant features. (b) T-SNE visualization for SimCLR.
• Figure 2: Each representation $z_i$ consists of domain-irrelevant common features $c_i$ and domain-relevant features.
• Figure 3: T-SNE visualization of a multi-domain 3D toy example. (a) and (d) are the toy example colored by domain and class, respectively. (b) and (e) are t-SNE visualizations for SimCLR. (c) and (f) are t-SNE visualizations for DCon.
• Figure 4: The framework for Prototype Mixup Learning.
• Figure 5: Comparison of t-SNE visualization on PACS. The source train domains includes art painting, cartoon, sketch. (a) Our proposed method on train samples with domain labels. (b) Our proposed method on train samples with class labels. T-SNE visualization for SimCLR is in Figure 1b.