Semi-Supervised Transfer Boosting (SS-TrBoosting)

TL;DR

SS-TrBoosting introduces a boosting-based fine-tuning framework that enhances deep domain adaptation by stacking multiple blocks, each containing a supervised DA learner and an SSL learner. It leverages random nonlinear feature mappings, data re-weighting to reduce domain bias, and SSL data augmentation, and can be extended to SS-SFDA through synthetic source data generation. Empirical results on DomainNet, Office-Home, and Office-31 show consistent improvements over a wide range of baselines for both SSDA and SFDA, with notable gains in low-resource (1-shot) settings and when extending UDA methods to SSDA. The approach is computationally efficient relative to standard ensemble methods and is compatible with a variety of existing DA techniques, indicating strong practical utility for real-world transfer learning with limited labeled target data.

Abstract

Semi-supervised domain adaptation (SSDA) aims at training a high-performance model for a target domain using few labeled target data, many unlabeled target data, and plenty of auxiliary data from a source domain. Previous works in SSDA mainly focused on learning transferable representations across domains. However, it is difficult to find a feature space where the source and target domains share the same conditional probability distribution. Additionally, there is no flexible and effective strategy extending existing unsupervised domain adaptation (UDA) approaches to SSDA settings. In order to solve the above two challenges, we propose a novel fine-tuning framework, semi-supervised transfer boosting (SS-TrBoosting). Given a well-trained deep learning-based UDA or SSDA model, we use it as the initial model, generate additional base learners by boosting, and then use all of them as an ensemble. More specifically, half of the base learners are generated by supervised domain adaptation, and half by semi-supervised learning. Furthermore, for more efficient data transmission and better data privacy protection, we propose a source data generation approach to extend SS-TrBoosting to semi-supervised source-free domain adaptation (SS-SFDA). Extensive experiments showed that SS-TrBoosting can be applied to a variety of existing UDA, SSDA and SFDA approaches to further improve their performance.

Figures (4)

• Figure 1: Illustration of the proposed SS-TrBoosting, which fine-tunes the initial deep learning classifier by adding a series of boosting blocks. The same features from the feature extractor are fed into each fine-tuning block as inputs.
• Figure 2: Structure of the $k$-th fine-tuning block. We combine labeled source data $(\bm{Z}^S,\bm{y}^S)$ and labeled target data $(\bm{Z}^T,\bm{y}^T)$ for supervised DA. The sample weights $\bm{w}$ and pseudo-labels $\tilde{\bm{y}}$ are obtained to train the base learners, as in LogitBoost. A consistency constraint is used in SSL, i.e., we add random noise $\bm{\varepsilon}$ to unlabeled target data $\bm{Z}^U$, and use pseudo-labels of $\bm{Z}^U$ to supervise the training of the augmented unlabeled target data.
• Figure 3: (a) Effect of random nonlinear feature mapping. "dim" is the dimensionality of the mapped features. (b) Effect of removing misclassified source data. $\bm{z}^S$ denotes source data. (c) Sensitivity to the noise magnitude $\xi$. The baseline was MME. (d) Sensitivity to the number of fine-tuning blocks $K$.
• Figure 4: t-SNE feature visualization of the synthesized source data and the true source data for a 31-way classification task D$\rightarrow$W on Office-31.