Guidance Not Obstruction: A Conjugate Consistent Enhanced Strategy for Domain Generalization

TL;DR

This work tackles domain generalization by arguing that marginal domain alignment is insufficient due to polymorphic domain-related class clusters within each class. It introduces Con2EM, a distribution-level module that treats domain-related clusters as hyper-instances and uses a distribution-level Universum to generate diverse $P^d(X|y)$ distributions, thereby enriching the training signal without explicit target-domain labels. A distribution statistics branch, a distribution-level classifier with kernel embedding, and Universum-based augmentation enable resampling from generated distributions to reinforce the instance-level classifier, with a PAC-Bayesian style bound supporting the approach. Empirical results on six benchmarks show Con2EM achieving competitive or state-of-the-art accuracy with lower computational cost than many baselines, especially when using a larger batch variant Con2EM-L.

Abstract

Domain generalization addresses domain shift in real-world applications. Most approaches adopt a domain angle, seeking invariant representation across domains by aligning their marginal distributions, irrespective of individual classes, naturally leading to insufficient exploration of discriminative information. Switching to a class angle, we find that multiple domain-related peaks or clusters within the same individual classes must emerge due to distribution shift. In other words, marginal alignment does not guarantee conditional alignment, leading to suboptimal generalization. Therefore, we argue that acquiring discriminative generalization between classes within domains is crucial. In contrast to seeking distribution alignment, we endeavor to safeguard domain-related between-class discrimination. To this end, we devise a novel Conjugate Consistent Enhanced Module, namely Con2EM, based on a distribution over domains, i.e., a meta-distribution. Specifically, we employ a novel distribution-level Universum strategy to generate supplementary diverse domain-related class-conditional distributions, thereby enhancing generalization. This allows us to resample from these generated distributions to provide feedback to the primordial instance-level classifier, further improving its adaptability to the target-agnostic. To ensure generation accuracy, we establish an additional distribution-level classifier to regularize these conditional distributions. Extensive experiments have been conducted to demonstrate its effectiveness and low computational cost compared to SOTAs.

Figures (6)

• Figure 1: Illustration of two angles for DG. Different colors represent different domains, and different border lines represent different classes. From the domain angle, most existing approaches focus on their marginal distributions. From the class angle, multiple domain-related peaks have emerged within the same individual classes.
• Figure 2: Illustration of two failed cases in DIR. The solid line squares denote the images sampled from the domains and the dashed squares denote the latent representations.
• Figure 3: Visualization of polymorphic domain-related clusters within each individual classes testing in Sketch on PACS using t-SNE. Different colors correspond to different domains and different shapes correspond to different classes.
• Figure 4: Illustration of a pipeline during training. Our Con2EM can be regarded as an additional module, which contains three components: 1) Distribution Statistics Module, 2) Augmentation Manipulation (AUG), and 3) Distribution-Level Classifier.
• Figure 5: Visualization of learned representations on PACS using t-SNE. Different colors correspond to different classes and different shapes correspond to different domains. (A)-(E) and (F)-(J) select Sketch and Cartoon as target domain, respectively.

Theorems & Definitions (1)

• Theorem 1