LFME: A Simple Framework for Learning from Multiple Experts in Domain Generalization

TL;DR

This work introduces a simple yet effective framework, dubbed learning from multiple experts (LFME), that aims to make the target model an expert in all source domains to improve DG.

Abstract

Domain generalization (DG) methods aim to maintain good performance in an unseen target domain by using training data from multiple source domains. While success on certain occasions are observed, enhancing the baseline across most scenarios remains challenging. This work introduces a simple yet effective framework, dubbed learning from multiple experts (LFME), that aims to make the target model an expert in all source domains to improve DG. Specifically, besides learning the target model used in inference, LFME will also train multiple experts specialized in different domains, whose output probabilities provide professional guidance by simply regularizing the logit of the target model. Delving deep into the framework, we reveal that the introduced logit regularization term implicitly provides effects of enabling the target model to harness more information, and mining hard samples from the experts during training. Extensive experiments on benchmarks from different DG tasks demonstrate that LFME is consistently beneficial to the baseline and can achieve comparable performance to existing arts. Code is available at~\url{https://github.com/liangchen527/LFME}.

Figures (6)

• Figure 1: Pipeline of LFME. Experts and the target model are trained simultaneously. To obtain a target model that is an expert on all source domains, we learn multiple experts specialized in corresponding domains to help guide the target model during training. For each sample, the guidance is implemented with a logit regularization term that enforces similarity between the logit of the target model and probability from the corresponding expert. Only the target model is utilized in inference. Please refer to Algorithm \ref{['alg']} for detailed implementations.
• Figure 2: Values of probabilities, logits, and rescaling factors(i.e.$q$, $z$, $\mathcal{F}$, $\mathcal{F}'$) from the ERM model and LFME. Models are trained on three source domains from PACS with the same settings.
• Figure 3: Qualitative comparisons. The compared methods make unsatisfactory predictions about several objects, such as clouds with varying shapes, car logo, or people and car in the shadow. Please zoom in for a better view.
• Figure 4: Grad-CAM visualizations of samples from the unseen "cartoon" domain of the PACS benchmark, which is the most challenging domain for ERM and our method according to Tab. \ref{['tab deepna']}. Compared to the baseline ERM, highlight regions from our method contain more information related to the label category. These visualizations can further validate our analysis in Sec. \ref{['sec deepana']} that with the proposed strategy, the target model can explore more information for prediction.
• Figure 5: Classification ratio comparisons of ERM and LFME in the hard and easy samples from the difficult TerraInc dataset. The closer the ratio $\mathcal{R}$ approaches 1, the better the corresponding prediction. Here the hard samples are specified by the experts: the $1/3$ samples in a training batch with larger losses from the experts, and the easy samples are the leading $1/3$ samples with smaller losses. The two models perform evenly well on the easy samples, while LFME obtains better results than ERM in the hard samples.