MLNet: Mutual Learning Network with Neighborhood Invariance for Universal Domain Adaptation

TL;DR

In this paper, a novel Mutual Learning Network with neighborhood invariance for UniDA is proposed, designed to reduce the intra-domain variations for more generalizable feature representation and compensates for the misidentified known-class errors by mutual learning between the closed-set and open-set classifiers.

Abstract

Universal domain adaptation (UniDA) is a practical but challenging problem, in which information about the relation between the source and the target domains is not given for knowledge transfer. Existing UniDA methods may suffer from the problems of overlooking intra-domain variations in the target domain and difficulty in separating between the similar known and unknown class. To address these issues, we propose a novel Mutual Learning Network (MLNet) with neighborhood invariance for UniDA. In our method, confidence-guided invariant feature learning with self-adaptive neighbor selection is designed to reduce the intra-domain variations for more generalizable feature representation. By using the cross-domain mixup scheme for better unknown-class identification, the proposed method compensates for the misidentified known-class errors by mutual learning between the closed-set and open-set classifiers. Extensive experiments on three publicly available benchmarks demonstrate that our method achieves the best results compared to the state-of-the-arts in most cases and significantly outperforms the baseline across all the four settings in UniDA. Code is available at https://github.com/YanzuoLu/MLNet.

Figures (14)

• Figure 1: CAM selvaraju2017grad and t-SNE van2008visualizing visualizations on the VisDA peng2017visda dataset. Our method outperforms the OVANet baseline by learning more discriminative features for both known and unknown classes even though they are similar. More t-SNE examples are provided in supplementary.
• Figure 2: During testing, decisions are made by comparing the positive score with the decision threshold (i.e., 0.5). Only the open-set classifier scores of the maximum-probability known class are used to decide between known or unknown.
• Figure 3: Schematics of MLNet. (a-d) Illustrating the effect of different losses. (e) Neighborhood is adaptive to the sample size of different classes. (f) Neighbors with more similar neighborhoods have higher confidence. (g) We simulate unknown-class samples across domains and utilize them to supervise the open-set classifiers for better unknown-class identification. (h) By optimizing $\mathcal{L}_{cc}$, misidentified known-class samples are corrected while unknown-class identification is not affected.
• Figure 4: Ablation studies for (a) consistency constraint and (b-d) hyperparameter sensitivity on the Office dataset. (a) Each row collects results for all settings with a specific target domain (e.g., Amazon with sample size of 94) when varying the loss weight $\eta$. A higher average ranking (up to 1) indicates higher performance across the 8 experiments (4 DA settings and 2 sources).
• Figure 5: Sensitivity analysis when varying the loss weight $\beta_1$ for neighborhood invariance learning on the Office dataset.

Theorems & Definitions (2)

• proof
• proof