Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Negatives-Dominant Contrastive Learning for Generalization in Imbalanced Domains

Meng Cao, Jiexi Liu, Songcan Chen

TL;DR

This work tackles Imbalanced Domain Generalization (IDG), where both domain and label shifts distort decision boundaries. It derives a novel generalization bound based on $ ext{H-divergence}$ that highlights the roles of posterior discrepancy and margin, motivating a shift from marginal alignment to boundary-focused learning. The authors propose Negative-Dominant Contrastive Learning (NDCL), which emphasizes negatives in a prediction-space InfoNCE-style objective, augments hard negatives, reweights intra-class samples, and enforces prediction-centered posterior alignment across domains. Through extensive experiments on VLCS, PACS, and OfficeHome under varied imbalance settings, NDCL consistently outperforms strong baselines and exhibits strong correlations between margin, posterior similarity, and target-domain accuracy, underscoring practical impact for robust cross-domain generalization under imbalance. Code is available at the project site, and the approach offers a principled path toward more generalizable models in real-world, imbalanced domain scenarios.

Abstract

Imbalanced Domain Generalization (IDG) focuses on mitigating both domain and label shifts, both of which fundamentally shape the model's decision boundaries, particularly under heterogeneous long-tailed distributions across domains. Despite its practical significance, it remains underexplored, primarily due to the technical complexity of handling their entanglement and the paucity of theoretical foundations. In this paper, we begin by theoretically establishing the generalization bound for IDG, highlighting the role of posterior discrepancy and decision margin. This bound motivates us to focus on directly steering decision boundaries, marking a clear departure from existing methods. Subsequently, we technically propose a novel Negative-Dominant Contrastive Learning (NDCL) for IDG to enhance discriminability while enforce posterior consistency across domains. Specifically, inter-class decision-boundary separation is enhanced by placing greater emphasis on negatives as the primary signal in our contrastive learning, naturally amplifying gradient signals for minority classes to avoid the decision boundary being biased toward majority classes. Meanwhile, intra-class compactness is encouraged through a re-weighted cross-entropy strategy, and posterior consistency across domains is enforced through a prediction-central alignment strategy. Finally, rigorous yet challenging experiments on benchmarks validate the effectiveness of our NDCL. The code is available at https://github.com/Alrash/NDCL.

Negatives-Dominant Contrastive Learning for Generalization in Imbalanced Domains

TL;DR

This work tackles Imbalanced Domain Generalization (IDG), where both domain and label shifts distort decision boundaries. It derives a novel generalization bound based on that highlights the roles of posterior discrepancy and margin, motivating a shift from marginal alignment to boundary-focused learning. The authors propose Negative-Dominant Contrastive Learning (NDCL), which emphasizes negatives in a prediction-space InfoNCE-style objective, augments hard negatives, reweights intra-class samples, and enforces prediction-centered posterior alignment across domains. Through extensive experiments on VLCS, PACS, and OfficeHome under varied imbalance settings, NDCL consistently outperforms strong baselines and exhibits strong correlations between margin, posterior similarity, and target-domain accuracy, underscoring practical impact for robust cross-domain generalization under imbalance. Code is available at the project site, and the approach offers a principled path toward more generalizable models in real-world, imbalanced domain scenarios.

Abstract

Imbalanced Domain Generalization (IDG) focuses on mitigating both domain and label shifts, both of which fundamentally shape the model's decision boundaries, particularly under heterogeneous long-tailed distributions across domains. Despite its practical significance, it remains underexplored, primarily due to the technical complexity of handling their entanglement and the paucity of theoretical foundations. In this paper, we begin by theoretically establishing the generalization bound for IDG, highlighting the role of posterior discrepancy and decision margin. This bound motivates us to focus on directly steering decision boundaries, marking a clear departure from existing methods. Subsequently, we technically propose a novel Negative-Dominant Contrastive Learning (NDCL) for IDG to enhance discriminability while enforce posterior consistency across domains. Specifically, inter-class decision-boundary separation is enhanced by placing greater emphasis on negatives as the primary signal in our contrastive learning, naturally amplifying gradient signals for minority classes to avoid the decision boundary being biased toward majority classes. Meanwhile, intra-class compactness is encouraged through a re-weighted cross-entropy strategy, and posterior consistency across domains is enforced through a prediction-central alignment strategy. Finally, rigorous yet challenging experiments on benchmarks validate the effectiveness of our NDCL. The code is available at https://github.com/Alrash/NDCL.
Paper Structure (32 sections, 4 theorems, 35 equations, 19 figures, 12 tables, 1 algorithm)

This paper contains 32 sections, 4 theorems, 35 equations, 19 figures, 12 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Methodology
  3. Preliminaries
  4. Theoretical Guarantee
  5. Negative-Dominant Contrastive Learning
  6. Experiments
  7. Datasets and Settings
  8. Results and Analyses
  9. Overall result analysis
  10. Ablation studies
  11. Margin and Posterior Discrepancy Analysis
  12. Parameter studies
  13. Extended Experimental Analyses
  14. Conclusion
  15. Error Bound for Imbalanced Domain Generalization
  16. ...and 17 more sections

Key Result

Theorem 1

Given $N$ domains with joint distributions $\left\{P_S^d(\bm{X}, \bm{Y})\right\}_{d=1}^{N}$, the target risk $\epsilon_T(h)$ for any hypothesis $h \in \mathcal{H}$ is defined via their linear mixture. To capture prediction-induced discrepancies, the classical $\mathcal{H}$-divergence is reformulated where $\sum_{d=1}^{N} \pi_d = 1$, and $\sum_{d=1}^{N}\pi_dP_S^d\left(\bm{X}, \bm{Y}\right)$ denotes

Figures (19)

  • Figure 1: Impact of Domain and Label Shifts on Decision Boundaries. Orange, blue, and peach represent three distinct domains. (A) Domain shift leads to domain-specific class boundaries (dashed lines) for each class. (B) Label shift causes majority classes to dominate, compressing margins for minority classes. The dashed and solid lines are equivalent, with the former included for visual consistency. (C) Their interaction amplifies the challenge, requiring robustness to both simultaneously.
  • Figure 2: Long-Tailed Positives vs. Relatively Balanced yet Abundant Negatives.
  • Figure 3: An illustration of NDCL, composed of three sub-objectives operating in the prediction space. (A) A contrastive loss that pushes away nearby negatives to enhance inter-class decision-boundary separation. (B) A class-wise re-weighted cross-entropy loss that encourages intra-class compactness and implictly aligns posteriors. Longer arrows indicate higher weights. (C).A prediction-central alignment strategy that explicitly promotes posterior consistency across domains.
  • Figure 4: Illustrative per-class comparison of margin $\gamma\left(h\right)$ and posterior discrepancy based on JS divergence on OfficeHome under two different settings across several representative methods including ours. $\left( \uparrow \right)$ denotes the larger the better, while $\left(\downarrow\right)$ denotes the opposite.
  • Figure 5: Joint influence of hyperparameters $\alpha$ and $\beta$ on OfficeHome under two different settings. The upper part corresponds to the TotalHeavyTail setting, and the lower part to the Duality setting. Log-scale axes are used, e.g., $0=\log_{10}1$.
  • ...and 14 more figures

Theorems & Definitions (7)

  • Theorem 1: Generalization Bound for Imbalanced Domain Generalization
  • Lemma 1
  • Proof 1
  • Theorem 2: Generalization Bound with the Joint Distributions
  • Theorem 3: Generalization Bound with the Joint Distributions on multiple source domains
  • Proof 2
  • Remark 1: Bounding the $\mathcal{H}$-divergence between domains in the convex hull albuquerque2019generalizing