Towards Generalizing to Unseen Domains with Few Labels

TL;DR

This work tackles semi-supervised domain generalization (SSDG), where abundant unlabeled data from multiple source domains must complement limited labeled data to generalize to unseen domains. The authors introduce a plug-and-play framework that augments SSL baselines with two losses: feature-based conformity (FBC), which aligns feature-space posteriors with pseudo-labels using domain-aware prototypes $\mathbf{K}_c^d$, and a semantics alignment (SA) constraint that regularizes the semantic layout by promoting cohesion around assigned prototypes and discouraging proximity to hard non-assigned prototypes. The method is model-agnostic and integrates with strong SSL baselines (e.g., FixMatch, StyleMatch) across five DG benchmarks, achieving consistent improvements (often several percentage points) in both 5- and 10-label settings. The results demonstrate improved cross-domain robustness in practical SSDG scenarios without adding extra learnable parameters, while noting limitations for semi-supervised single-source DG and suggesting avenues for extending the framework.

Abstract

We approach the challenge of addressing semi-supervised domain generalization (SSDG). Specifically, our aim is to obtain a model that learns domain-generalizable features by leveraging a limited subset of labelled data alongside a substantially larger pool of unlabeled data. Existing domain generalization (DG) methods which are unable to exploit unlabeled data perform poorly compared to semi-supervised learning (SSL) methods under SSDG setting. Nevertheless, SSL methods have considerable room for performance improvement when compared to fully-supervised DG training. To tackle this underexplored, yet highly practical problem of SSDG, we make the following core contributions. First, we propose a feature-based conformity technique that matches the posterior distributions from the feature space with the pseudo-label from the model's output space. Second, we develop a semantics alignment loss to learn semantically-compatible representations by regularizing the semantic structure in the feature space. Our method is plug-and-play and can be readily integrated with different SSL-based SSDG baselines without introducing any additional parameters. Extensive experimental results across five challenging DG benchmarks with four strong SSL baselines suggest that our method provides consistent and notable gains in two different SSDG settings.

Figures (5)

• Figure 1: Visual illustration of Semi-supervised Domain Generalization (SSDG) setting.
• Figure 2: Recognition performance comparison between different DG, DG combined with pseudo-labeling, SSL methods and ours in SSDG settings. Here, GD - GroupDro, FlexM - FlexMatch, FreeM - FreeMatch, FixM - FixMatch and StyleM - StyleMatch.
• Figure 3: PL accuracy in the training samples without thresholding (left) and for selected PL after thresholding (right) for the baseline (FixMatch sohn2020fixmatch), ours with only feature-based conformity, and ours for PACS dataset in 5 labels per class setting. Here A, C, P, and S denote Art-painting, Cartoon, Photos, and Sketch domains, respectively.
• Figure 4: Overall architecture of our method. Fundamentally, it is a semi-supervised baseline (e.g., FixMatch sohn2020fixmatch) with a feature extractor and a classifier that involves pseudo-labelling and prediction consistency mechanisms. To tackle semi-supervised domain generalization, we first propose a feature-based conformity module (sec. \ref{['subsec:Feature_based_conformity']}) that aligns the posterior from feature space with the pseudo-label from output space. We then develop a semantics alignment loss (sec. \ref{['subsec: sematic_alignment_constraint']}) to regularize the semantic layout of feature space and further improve the effectiveness of feature-based conformity.
• Figure 5: We visualize the feature space using tSNE (left), and the cosine similarity between the means of class-wise features (right) for PACS test domains of baseline (first row), ours with feature-based conformity only (second row), and ours with both feature-based conformity and semantic alignment loss (third row).