SepRep-Net: Multi-source Free Domain Adaptation via Model Separation And Reparameterization

TL;DR

A novel framework called SepRep-Net is proposed, which tackles multi-source free domain adaptation via model Separation and Reparameterization, and is characterized by effectiveness: competitive performance on the target domain, efficiency: low computational costs, and generalizability: maintaining more source knowledge than existing solutions.

Abstract

We consider multi-source free domain adaptation, the problem of adapting multiple existing models to a new domain without accessing the source data. Among existing approaches, methods based on model ensemble are effective in both the source and target domains, but incur significantly increased computational costs. Towards this dilemma, in this work, we propose a novel framework called SepRep-Net, which tackles multi-source free domain adaptation via model Separation and Reparameterization.Concretely, SepRep-Net reassembled multiple existing models to a unified network, while maintaining separate pathways (Separation). During training, separate pathways are optimized in parallel with the information exchange regularly performed via an additional feature merging unit. With our specific design, these pathways can be further reparameterized into a single one to facilitate inference (Reparameterization). SepRep-Net is characterized by 1) effectiveness: competitive performance on the target domain, 2) efficiency: low computational costs, and 3) generalizability: maintaining more source knowledge than existing solutions. As a general approach, SepRep-Net can be seamlessly plugged into various methods. Extensive experiments validate the performance of SepRep-Net on mainstream benchmarks.

Figures (3)

• Figure 1: Problem Setup and Method Performance. (a) We reassemble multiple existing source models to one target model that is adapted to a novel unlabeled target domain. The target model is expected to be 1) effective for target data, 2) efficient in inference and 3) able to generalize well on source data, i.e. preserving more source knowledge. (b) Compared to previous methods on Office-Home, our framework enjoys better effectiveness, generalizability, and efficiency (H-score $\frac{2 \cdot Acc_{source} \cdot Acc_{target}}{Acc_{source} + Acc_{target}}$ ($\%$) evaluates effectiveness and generalizability jointly).
• Figure 2: Method Overview. Take three source models as an example. Train: In each Conv-BN unit, separate pathways are applied in parallel, forming a structure with multiple pathways. The outputs are then integrated into a unified output via the feature merging unit. Inference: Multiple pathways are re-parameterized to a unique one during inference. An uncertainty-based weighting strategy ensembles multiple classifier heads to obtain the final prediction. (Best viewed in color)
• Figure 3: Output Weights for different domains in Office-Home with ResNet-50 backbone when applying SepRep-Net to SHOT. (a) The output weights calculated in our framework are positively correlated to the classifier accuracy. (b) Our importance weights are adaptive to the input. Specifically, the importance weight of the input domain adaptively becomes larger than others, enhancing generalizability.