Self-Supervised Graph Neural Network for Multi-Source Domain Adaptation

TL;DR

This work addresses multi-source domain adaptation by leveraging unlabeled target data through self-supervision. It introduces Self-Supervised Graph Neural Network (SSG), which uses a graph with domain and category nodes and a graph neural network to enable direct information exchange between the domain-aware pretext task and the category prediction task, augmented by a mask token strategy to enhance domain representations. The model optimizes a multi-task objective that combines supervised category loss with a domain classification loss, while constructing adjacency via a Gaussian kernel and updating embeddings through graph convolutions. Empirically, SSG achieves state-of-the-art results on four MSDA benchmarks (Office-Caltech10, Office-31, Office-Home, DomainNet), with ablations demonstrating the effectiveness of the graph bridge, the mask token strategy, and the joint loss terms for robust cross-domain generalization.

Abstract

Domain adaptation (DA) tries to tackle the scenarios when the test data does not fully follow the same distribution of the training data, and multi-source domain adaptation (MSDA) is very attractive for real world applications. By learning from large-scale unlabeled samples, self-supervised learning has now become a new trend in deep learning. It is worth noting that both self-supervised learning and multi-source domain adaptation share a similar goal: they both aim to leverage unlabeled data to learn more expressive representations. Unfortunately, traditional multi-task self-supervised learning faces two challenges: (1) the pretext task may not strongly relate to the downstream task, thus it could be difficult to learn useful knowledge being shared from the pretext task to the target task; (2) when the same feature extractor is shared between the pretext task and the downstream one and only different prediction heads are used, it is ineffective to enable inter-task information exchange and knowledge sharing. To address these issues, we propose a novel \textbf{S}elf-\textbf{S}upervised \textbf{G}raph Neural Network (SSG), where a graph neural network is used as the bridge to enable more effective inter-task information exchange and knowledge sharing. More expressive representation is learned by adopting a mask token strategy to mask some domain information. Our extensive experiments have demonstrated that our proposed SSG method has achieved state-of-the-art results over four multi-source domain adaptation datasets, which have shown the effectiveness of our proposed SSG method from different aspects.

Figures (5)

• Figure 1: This figure shows the general framework of our proposed self-supervised graph neural network method. The blue nodes represent the embedding of categories in the target task, and the yellow nodes represent the embedding of domains in the self-supervised task. The graph neural network establishes the connection between the self-supervised task and the target task.
• Figure 2: This figure shows the overall framework of our proposed method. The left half exhibits the traditional multi-task self-supervised learning framework by adopting a shared feature extractor and different heads. The right half displays our proposed method by using a graph neural network to exchange information between the pretext task and the target task. The embedding selection in the lower-right shows the mask domain process. We will randomly select a representation of domain embeddings for the graph neural network. The red one represents that this image belongs to this domain, while the green one indicates that this image does not belong to this domain.
• Figure 3: This figure shows the dataset we adopted for the experiments. The upper-left is the Office-Caltech10 dataset. The lower-left is the Office-31 dataset. The upper-right is the Office-Home dataset. The lower-right is the DomainNet dataset.
• Figure 4: We use blue to represent our graph-based method and red to represent the linear-based method. Three line graphs on the left represent the trends of different loss functions, which is conducted in Office-31 with the target domain Webcam of 500 epochs. The histogram on the right represents the overall accuracy of the two methods on Office-31 and Office-Home.
• Figure 5: We study mask ratio and draw a line graph. The upper-left is the change of self-supervised loss and the upper-right is the domain accuracy in training stage. The lower-left is the category accuracy during training stage and the lower-right is the final accuracy in test phrase. All experiments are conducted in Office-31 with the target domain Webcam of 500 epochs.