Feature Modulation for Semi-Supervised Domain Generalization without Domain Labels

TL;DR

This work tackles semisupervised domain generalization (SSDG) without access to domain labels during training. It introduces a feature modulator guided by Similar Average Representations (SAR) to suppress domain-specific information and emphasize class-discriminative signals, paired with an uncertainty-aware, loss-scaled pseudo-labeling scheme to better utilize unlabeled data. The method achieves consistent gains over FixMatch baselines across four major DG benchmarks (PACS, OfficeHome, VLCS, DigitsDG) in both 5- and 10-label settings, and demonstrates improved class separation with robust performance under class imbalance. By removing the need for domain labels while boosting pseudo-label quality and unlabeled data usage, the approach has practical implications for real-world domain generalization tasks where domain annotations are scarce or unavailable.

Abstract

Semi-supervised domain generalization (SSDG) leverages a small fraction of labeled data alongside unlabeled data to enhance model generalization. Most of the existing SSDG methods rely on pseudo-labeling (PL) for unlabeled data, often assuming access to domain labels-a privilege not always available. However, domain shifts introduce domain noise, leading to inconsistent PLs that degrade model performance. Methods derived from FixMatch suffer particularly from lower PL accuracy, reducing the effectiveness of unlabeled data. To address this, we tackle the more challenging domain-label agnostic SSDG, where domain labels for unlabeled data are not available during training. First, we propose a feature modulation strategy that enhances class-discriminative features while suppressing domain-specific information. This modulation shifts features toward Similar Average Representations-a modified version of class prototypes-that are robust across domains, encouraging the classifier to distinguish between closely related classes and feature extractor to form tightly clustered, domain-invariant representations. Second, to mitigate domain noise and improve pseudo-label accuracy, we introduce a loss-scaling function that dynamically lowers the fixed confidence threshold for pseudo-labels, optimizing the use of unlabeled data. With these key innovations, our approach achieves significant improvements on four major domain generalization benchmarks-even without domain labels. We will make the code available.

Figures (6)

• Figure 1: Feature modulation towards SARs in the classwise separated feature space. Polygon lines denote class boundaries. A: Class prototypes $P_c$ in the feature space. B: SAR of class C (Sec. \ref{['Similar Average Representations']}) generated by averaging class prototypes based on similarity. The glow intensity represents the similarity. C: Instance-specific features $z$ in the feature space with SARs of all classes. D: Feature modulation shifts $z$ more towards its true class.
• Figure 2: Average accuracy, keep rate, and pseudo-label (PL) accuracy. We achieve higher average accuracy as a result of the increased PL accuracy and higher keep rate. Note that different $y$-axis ranges were chosen to enhance the visualization.
• Figure 3: Proposed Architecture: a) Similar Average Representation (SAR (Sec. \ref{['Similar Average Representations']})) generation. b) Complete model during training. c) Feature Modulator (Sec. \ref{['Feature Modulation']}) combines instance-specific features ($z$) and SAR ($\mathbf{R}$) to soft-eliminate domain noise using our modulation strategy. This results in modulated features $\mathbf{Z}_m$. We input this $\mathbf{Z}_m$ to the shared classifier $\mathcal{C}$ and get the probabilities for each class. With these probabilities, we compute the MSE and NLL losses for both labeled and unlabeled data. For unsupervised losses, we consider loss scaling (Sec. \ref{['uncertanity_and_loss_scale']}) to maximize unlabeled data utilization. Note that our method is agnostic of domain labels.
• Figure 4: Comparison of PL accuracy and keep rate with training iterations (20) for target domain Art of OfficeHome dataset. We achieve consistent improvements over FixMatch.
• Figure 5: t-SNE visualization of features of three source domains (Cartoon, Photo, and Sketch) in the PACS dataset after training. The separation has gained improvements for class labels 0, 2, 4, and 5. (dog, giraffe, horse, and house respectively). Left: Features from feature extractor in FixMatch. Right: Modulated features in our method.