Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Connecting Domains and Contrasting Samples: A Ladder for Domain Generalization

Tianxin Wei, Yifan Chen, Xinrui He, Wenxuan Bao, Jingrui He

TL;DR

This work tackles domain generalization by diagnosing the failure of self-contrastive learning to transfer across domains due to weak intra-class connectivity. It introduces domain-connecting contrastive learning (DCCL), which strengthens cross-domain intra-class links through aggressive data augmentation, cross-domain positive samples, pre-trained model anchoring, and a variational generative transformation loss. The method combines an enhanced contrastive objective with supervised learning, yielding state-of-the-art out-of-domain performance across five benchmarks and demonstrating robust gains under limited labels and varying backbones. The contributions provide a practical pathway to improve DG without relying on explicit domain supervision, with broad implications for real-world generalization under distribution shifts.

Abstract

Distribution shifts between training and testing samples frequently occur in practice and impede model generalization performance. This crucial challenge thereby motivates studies on domain generalization (DG), which aim to predict the label on unseen target domain data by solely using data from source domains. It is intuitive to conceive the class-separated representations learned in contrastive learning (CL) are able to improve DG, while the reality is quite the opposite: users observe directly applying CL deteriorates the performance. We analyze the phenomenon with the insights from CL theory and discover lack of intra-class connectivity in the DG setting causes the deficiency. We thus propose a new paradigm, domain-connecting contrastive learning (DCCL), to enhance the conceptual connectivity across domains and obtain generalizable representations for DG. On the data side, more aggressive data augmentation and cross-domain positive samples are introduced to improve intra-class connectivity. On the model side, to better embed the unseen test domains, we propose model anchoring to exploit the intra-class connectivity in pre-trained representations and complement the anchoring with generative transformation loss. Extensive experiments on five standard DG benchmarks are performed. The results verify that DCCL outperforms state-of-the-art baselines even without domain supervision. The detailed model implementation and the code are provided through https://github.com/weitianxin/DCCL

Connecting Domains and Contrasting Samples: A Ladder for Domain Generalization

TL;DR

This work tackles domain generalization by diagnosing the failure of self-contrastive learning to transfer across domains due to weak intra-class connectivity. It introduces domain-connecting contrastive learning (DCCL), which strengthens cross-domain intra-class links through aggressive data augmentation, cross-domain positive samples, pre-trained model anchoring, and a variational generative transformation loss. The method combines an enhanced contrastive objective with supervised learning, yielding state-of-the-art out-of-domain performance across five benchmarks and demonstrating robust gains under limited labels and varying backbones. The contributions provide a practical pathway to improve DG without relying on explicit domain supervision, with broad implications for real-world generalization under distribution shifts.

Abstract

Distribution shifts between training and testing samples frequently occur in practice and impede model generalization performance. This crucial challenge thereby motivates studies on domain generalization (DG), which aim to predict the label on unseen target domain data by solely using data from source domains. It is intuitive to conceive the class-separated representations learned in contrastive learning (CL) are able to improve DG, while the reality is quite the opposite: users observe directly applying CL deteriorates the performance. We analyze the phenomenon with the insights from CL theory and discover lack of intra-class connectivity in the DG setting causes the deficiency. We thus propose a new paradigm, domain-connecting contrastive learning (DCCL), to enhance the conceptual connectivity across domains and obtain generalizable representations for DG. On the data side, more aggressive data augmentation and cross-domain positive samples are introduced to improve intra-class connectivity. On the model side, to better embed the unseen test domains, we propose model anchoring to exploit the intra-class connectivity in pre-trained representations and complement the anchoring with generative transformation loss. Extensive experiments on five standard DG benchmarks are performed. The results verify that DCCL outperforms state-of-the-art baselines even without domain supervision. The detailed model implementation and the code are provided through https://github.com/weitianxin/DCCL
Paper Structure (28 sections, 1 theorem, 10 equations, 5 figures, 16 tables)

This paper contains 28 sections, 1 theorem, 10 equations, 5 figures, 16 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Data in the Domain Generalization Setting
  4. Model Optimization with Contrastive Learning
  5. Proposed Methodology
  6. Motivation: Failure of Self-contrastive Learning in Domain Generalization
  7. Implications from Contrastive Learning Theory
  8. More Aggressive Data Augmentation and Cross-domain Positive Samples
  9. Anchoring Learned Maps to Pre-trained Model
  10. Experiments
  11. Experimental Settings
  12. Results (RQ1)
  13. Ablation Studies (RQ2)
  14. Case Studies
  15. Related work
  16. ...and 13 more sections

Key Result

proposition 1

Under Assumptions 1 & 2, by minimizing the InfoNCE loss we can conclude that the conditional variance terms vanish at the minimizer $f^*$, i.e.,

Figures (5)

  • Figure 1: t-SNE visualization of the representations across both training and testing domains, output by Pre-trained, ERM, SCL and our DCCL respectively. Same-class points share colors, while marker types differentiate training and testing domains. (Please zoom in for better viewing.) We visualize the embedding on PACS dataset where the source domains are [Photo], [Sketch], and [Cartoon]; the target domain is [Art]. Note that when mapped by the pre-trained model, intra-class samples from both the training and testing domains appear scattered but indeed well-connected. SCL will lead to a degradation in the embedding quality. Our proposed DCCL, on the other hand, effectively clusters the intra-class samples.
  • Figure 2: The overall framework of DCCL. The green dotted arrows indicate the two representations form a positive pair and the red ones connect the negative pairs. $a(\cdot)$ is an aggressive augmentation operation. Two key parts in DCCL are (i) cross-domain data contrast to bridge the intra-class samples across domains; (ii) pre-trained model anchoring, completed with generative transformation to harness the intra-class connectivity inherent in the pre-trained representation.
  • Figure 3: Illustration for the toy example of self-contrastive learning (SCL). Spots and slashes are filled in to represent different domains; orange and black rectangles respectively denote classes 1 and 2. The mapping function $\varphi \circ \theta$ learned on domain $d_1$ can perfectly classify the samples, and the mapping attains perfect alignment and uniformity (the objective of SCL). When trained with weak augmentation and applied to a new domain $d_2$, the classifier completely fails (0% acc). With aggressive augmentation, the intra-class samples of different domains are connected and we obtain transferable representations (100% acc).
  • Figure 4: An overview of the generative transformation module in DCCL. Two representations $z_{pre}$ and $z$ of the same image are generated via the pre-trained and the fine-tuned model respectively. The variational reconstruction is conducted to encode essential within-sample information.
  • Figure 5: t-SNE visualization of the ERM, PCL and DCCL representations on the testing domain. Same-class points are in the same colors. We visualize the embedding on PACS dataset where the source domains are photo, sketch, and cartoon; the target domain is art.

Theorems & Definitions (3)

  • Example 3.1: SCL does not help domain generalization.
  • Definition C.1: Sample Connectivity wang2022chaos
  • proposition 1